Conditions for Applicability of Cross-Component Coding

ABSTRACT

A method for visual media processing, including: using a rule that specifies representing, based on a fixed value, a chroma residual scaling factor for at least one chroma block associated with a current video unit of visual media data, wherein the fixed value is based at least on one or more conditions; and performing a conversion between the current video unit and a bitstream representation of the current video unit.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No. 17/494,974, filed on Oct. 6, 2021, which is based on PCT/CN2020/089096, filed on May 8, 2020, which claims the priority to and benefit of International Patent Application No. PCT/CN2019/085996, filed on May 8, 2019. All the aforementioned patent applications are hereby incorporated by reference in their entireties.

TECHNICAL FIELD

This patent document relates to video coding/decoding techniques, devices and systems.

BACKGROUND

In spite of the advances in video compression, digital video still accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow.

SUMMARY

Devices, systems and methods related to digital video coding/decoding, and specifically, simplified linear model derivations for the cross-component linear model (CCLM) prediction mode in video coding/decoding are described. The described methods may be applied to both the existing video coding standards (e.g., High Efficiency Video Coding (HEVC)) and future video coding standards (e.g., Versatile Video Coding (VVC)) or codecs.

In one representative aspect, the disclosed embodiment includes a method for visual media processing. This exemplary method includes: in response to determining availability of neighboring luma blocks of a corresponding luma block of visual media data, deriving a first parameter for a current video chroma block of the visual media data such that the first parameter is derived as a fixed value or a rounding-based average value of the neighboring luma samples of the current video chroma block, wherein the corresponding luma block covers a location associated with the current video chroma block; determining a chroma scaling factor based on the first parameter; performing a chroma residual scaling of the current video chroma block based on the chroma scaling factor; and using the chroma scaling factor for a conversion between the current video chroma block and a bitstream representation of the current video chroma block.

In one representative aspect, the disclosed embodiment includes a method for visual media processing. This exemplary method includes using a rule that specifies representing, based on a fixed value, a chroma residual scaling factor for at least one chroma block associated with a current video unit of visual media data, wherein the fixed value is based at least on one or more conditions; and performing a conversion between the current video unit and a bitstream representation of the current video unit.

In one representative aspect, the disclosed embodiment includes a method for visual media processing. This exemplary method includes, during a conversion between a current video block of visual media data and a bitstream representation of the current video block, making a determination of selectively enabling or disabling application of a cross-component linear model (CCLM) on color components of the current video block, based at least in part on dimensions and/or a partition information and/or a coding structure type associated with the current video block, wherein, in the CCLM, a first set of color components are predicted based on a linear combination of a second set of color components.

In yet another example aspect, the above-described method may be implemented by a video encoder apparatus that comprises a processor.

In yet another example aspect, the above-described method may be implemented by a video decoder apparatus that comprises a processor.

In yet another example aspect, these methods may be embodied in the form of processor-executable instructions and stored on a computer-readable program medium.

These, and other, aspects are further described in the present document.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of angular intra prediction modes in HEVC.

FIG. 2 shows an example of directional modes not in HEVC.

FIG. 3 shows an example in connection with the CCLM mode.

FIG. 4 shows an example of luma mapping with chroma scaling architecture.

FIG. 5 shows an example of luma block and chroma block in different color formats.

FIG. 6 shows an example of luma block and chroma block in same color formats.

FIG. 7 shows an example of collocated luma block covering multiple formats.

FIG. 8 shows an example of luma block within a larger luma block.

FIG. 9 shows an example of luma block within a larger luma block and within a bounding box.

FIG. 10 is a block diagram of an example of a hardware platform for implementing a visual media decoding or a visual media encoding technique according to various embodiments of the disclosure.

FIG. 11 shows a flowchart of an example method for linear model derivations for cross-component prediction according to various embodiments of the disclosure.

FIG. 12 is a block diagram of an example video processing system in which disclosed embodiments may be implemented.

FIG. 13 is a flowchart for an example for a visual media processing method.

FIG. 14 is a flowchart for an example for a visual media processing method.

FIG. 15 is a flowchart for an example for a visual media processing method.

DETAILED DESCRIPTION 2.1 A Brief Review on HEVC 2.1.1 Intra Prediction in HEVC/H.265

Intra prediction involves producing samples for a given transform block (TB) using samples previously reconstructed in the considered color channel. The intra prediction mode is separately signaled for the luma and chroma channels, with the chroma channel intra prediction mode optionally dependent on the luma channel intra prediction mode via the ‘DM_CHROMA’ mode. Although the intra prediction mode is signaled at the PB (prediction block) level, the intra prediction process is applied at the TB level, in accordance with the residual quad-tree hierarchy for the coding unit (CU), thereby allowing the coding of one TB to have an effect on the coding of the next TB within the CU, and therefore reducing the distance to the samples used as reference values.

HEVC includes 35 intra prediction modes—a direct current (DC) mode, a planar mode and 33 directional, or ‘angular’ intra prediction modes. The 33 angular intra prediction modes are illustrated in FIG. 1 .

For prediction blocks (PBs) associated with chroma color channels, the intra prediction mode is specified as either planar, DC, horizontal, vertical, ‘DM_CHROMA’ mode or sometimes diagonal mode ‘34’.

Note for chroma formats 4:2:2 and 4:2:0, the chroma PB may overlap two or four (respectively) luma PBs; in this case the luma direction for DM_CHROMA is taken from the top left of these luma PBs.

The DM_CHROMA mode indicates that the intra prediction mode of the luma color channel PB is applied to the chroma color channel PBs. Since this is relatively common, the most-probable-mode coding scheme of the intra_chroma_pred_mode is biased in favor of this mode being selected.

2.2 Versatile Video Coding (VVC) Algorithm Description 2.2.1 VVC Coding Architecture

To explore the future video coding technologies beyond HEVC, Joint Video Exploration Team (JVET) was founded by video coding experts group (VCEG) and moving pictures experts group (MPEG) jointly in 2015. The JVET meeting is concurrently held once every quarter, and the new coding standard is targeting at 50% bitrate reduction as compared to HEVC. The new video coding standard was officially named as Versatile Video Coding (VVC) in the April 2018 JVET meeting, and the first version of VVC test model (VTM) was released at that time. As there are continuous effort contributing to VVC standardization, new coding techniques are being adopted to the VVC standard in every JVET meeting. The VVC working draft and test model VTM are then updated after every meeting. The VVC project is now aiming for technical completion (Final Draft International Standard (FDIS)) at the July 2020 meeting.

As in most preceding standards, VVC has a block-based hybrid coding architecture, combining inter-picture and intra-picture prediction and transform coding with entropy coding. The picture partitioning structure divides the input video into blocks called coding tree units (CTUs). A CTU is split using a quadtree with nested multi-type tree structure into coding units (CUs), with a leaf coding unit (CU) defining a region sharing the same prediction mode (e.g. intra or inter). In this document, the term ‘unit’ defines a region of an image covering all color components; the term ‘block’ is used to define a region covering a particular color component (e.g. luma), and may differ in spatial location when considering the chroma sampling format such as 4:2:0.

2.2.2 Dual/Separate Tree Partition in VVC

Luma component and chroma component can have separate partition trees for I slices. Separate tree partitioning is under 64×64 block level instead of CTU level. In VTM software, there is a sequence parameter set (SPS) flag to control the dual-tree on and off.

2.2.3 Intra Prediction in VVC 2.2.3.1 67 Intra Prediction Modes

To capture the arbitrary edge directions presented in natural video, the number of directional intra modes in VTM4 is extended from 33, as used in HEVC, to 65. The new directional modes not in HEVC are depicted as red dotted arrows in FIG. 2 , and the planar and DC modes remain the same. These denser directional intra prediction modes apply for all block sizes and for both luma and chroma intra predictions.

2.2.3.2 Cross-Component Linear Model Prediction (CCLM)

To reduce the cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in the VTM4, for which the chroma samples are predicted based on the reconstructed luma samples of the same CU by using a linear model as follows:

pred_(C)(i,j)=α·rec_(L)′(i,j)+β

where pred_(C)(i, j) represents the predicted chroma samples in a CU and rec_(L)(i, j) represents the downsampled reconstructed luma samples of the same CU. Linear model parameter α and β are derived from the relation between luma values and chroma values from two samples, which are luma sample with minimum sample value and with maximum sample inside the set of downsampled neighboring luma samples, and their corresponding chroma samples. The linear model parameters α and β are obtained according to the following equations.

$\alpha = \frac{Y_{a} - Y_{b}}{X_{a} - X_{b}}$ β = Y_(b) − α ⋅ X_(b)

Where Y_(a) and X_(a) represent luma value and chroma value of the luma sample with maximum luma sample value. And X_(b) and Y_(b) represent luma value and chroma value of the luma sample with minimum luma sample, respectively. FIG. 3 shows an example of the location of the left and above samples and the sample of the current block involved in the CCLM mode.

The division operation to calculate parameter α is implemented with a look-up table. To reduce the memory required for storing the table, the diff value (difference between maximum and minimum values) and the parameter α are expressed by an exponential notation. For example, diff is approximated with a 4-bit significant part and an exponent. Consequently, the table for 1/diff is reduced into 16 elements for 16 values of the significand as follows:

DivTable[ ]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}

This would have a benefit of both reducing the complexity of the calculation as well as the memory size required for storing the needed tables.

Besides the above template and left template can be used to calculate the linear model coefficients together, they also can be used alternatively in the other 2 LM modes, called LM_A, and LM_L modes.

In LM_A mode, only the above template is used to calculate the linear model coefficients. To get more samples, the above template is extended to (W+H). In LM_L mode, only left template are used to calculate the linear model coefficients. To get more samples, the left template are extended to (H+W).

For a non-square block, the above template is extended to W+W, the left template are extended to H+H.

To match the chroma sample locations for 4:2:0 video sequences, two types of downsampling filter are applied to luma samples to achieve 2 to 1 downsampling ratio in both horizontal and vertical directions. The selection of downsampling filter is specified by a SPS level flag. The two downsampling filters are as follows, which are corresponding to “type-0” and “type-2” content, respectively.

${{rec}_{L}^{\prime}\left( {i,j} \right)} = {\begin{bmatrix} \begin{matrix} {{{rec}_{L}\left( {{{2i} - 1},{{2j} - 1}} \right)} + {{2 \cdot {rec}_{L}}\left( {{{2i} - 1},{{2j} - 1}} \right)} +} \\ {{{rec}_{L}\left( {{{2i} + 1},{{2j} - 1}} \right)} +} \end{matrix} \\ {{{rec}_{L}\left( {{{2i} - 1},{2j}} \right)} + {{2 \cdot {rec}_{L}}\left( {{2i},{2j}} \right)} + {{rec}_{L}\left( {{{2i} + 1},{2j}} \right)} + 4} \end{bmatrix} \gg 3}$ ${{rec}_{L}^{\prime}\left( {i,j} \right)} = {\begin{bmatrix} {{{rec}_{L}\left( {{2i},{{2j} - 1}} \right)} + {{rec}_{L}\left( {{{2i} - 1},{2j}} \right)} + {4 \cdot {{rec}_{L}\left( {{2i},{2j}} \right)}} +} \\ {{{rec}_{L}\left( {{{2i} + 1},{2j}} \right)} + {{rec}_{L}\left( {{2i},{{2j} + 1}} \right)} + 4} \end{bmatrix} \gg 3}$

Note that only one luma line (general line buffer in intra prediction) is used to make the downsampled luma samples when the upper reference line is at the CTU boundary.

This parameter computation is performed as part of the decoding process, and is not just as an encoder search operation. As a result, no syntax is used to convey the α and β values to the decoder.

For chroma intra mode coding, a total of 8 intra modes are allowed for chroma intra mode coding. Those modes include five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, and LM_L). Chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partitioning structure for luma and chroma components is enabled in I slices, one chroma block may correspond to multiple luma blocks. Therefore, for Chroma DM mode, the intra prediction mode of the corresponding luma block covering the center position of the current chroma block is directly inherited.

2.2.3.2.1 Corresponding Modified Working Draft (JVET-N0271)

The following spec is based on the modified working draft of JVET-M1001 and the adoption in JVET-N0271. The modifications of the adopted JVET-N0220 are shown in bold and underlining.

Syntax table Sequence parameter set RBSP syntax  

 

   

   

 

   

   

 

 

   

 

   

   

 

   

 

Semantics

sps_cclm_enabled_flag equal to 0 specifies that the cross-component linear model intra prediction from luma component to chroma component is disabled. sps_cclm_enabled_flag equal to 1 specifies that the cross-component linear model intra prediction from luma component to chroma component is enabled.

Decoding Process

-   In 8.4.4.2.8 Specification of INTRA_LT_CCLM, INTRA_L_CCLM and     INTRA_T_CCLM intra prediction mode     Inputs to this process are:     -   the intra prediction mode predModeIntra,     -   a sample location (xTbC, yTbC) of the top-left sample of the         current transform block relative to the top-left sample of the         current picture,     -   a variable nTbW specifying the transform block width,     -   a variable nTbH specifying the transform block height,     -   chroma neighbouring samples p[x][y], with x=−1, y=0 . . .         2*nTbH−1 and x=0 . . . 2*nTbW−1, y=−1.         Output of this process are predicted samples predSamples[x][y],         with x=0 . . . nTbW−1, y=0 . . . nTbH−1.         The current luma location (xTbY, yTbY) is derived as follows:

(xTbY,yTbY)=(xTbC<<1,yTbC<<1)  (8-156)

The variables availL, availT and availTL are derived as follows:

-   -   The availability of left neighbouring samples derivation process         for a block as specified in clause 6.4.X [Ed (BB): Neighbouring         blocks availability checking process tbd] is invoked with the         current chroma location (xCurr, yCurr) set equal to (xTbC, yTbC)         and the neighbouring chroma location (xTbC−1, yTbC) as inputs,         and the output is assigned to availL.     -   The availability of top neighbouring samples derivation process         for a block as specified in clause 6.4.X [Ed (BB): Neighbouring         blocks availability checking process tbd] is invoked with the         current chroma location (xCurr, yCurr) set equal to (xTbC, yTbC)         and the neighbouring chroma location (xTbC, yTbC−1) as inputs,         and the output is assigned to availT.     -   The availability of top-left neighbouring samples derivation         process for a block as specified in clause 6.4.X [Ed (BB:         Neighbouring blocks availability checking process tbd] is         invoked with the current chroma location (xCurr, yCurr) set         equal to (xTbC, yTbC) and the neighbouring chroma location         (xTbC−1, yTbC−1) as inputs, and the output is assigned to         availTL.     -   The number of available top-right neighbouring chroma samples         numTopRight is derived as follows:         -   The variable numTopRight is set equal to 0 and availTR is             set equal to TRUE.         -   When predModeIntra is equal to INTRA_T_CCLM, the following             applies for x=nTbW . . . 2*nTbW−1 until availTR is equal to             FALSE or x is equal to 2*nTbW−1:             -   The availability derivation process for a block as                 specified in clause 6.4.X [Ed (BB): Neighbouring blocks                 availability checking process tbd] is invoked with the                 current chroma location (xCurr, yCurr) set equal to                 (xTbC, yTbC) and the neighbouring chroma location                 (xTbC+x, yTbC−1) as inputs, and the output is assigned                 to availableTB             -   When availableTR is equal to TRUE, numTopRight is                 incremented by one.     -   The number of available left-below neighbouring chroma samples         numLeftBelow is derived as follows:         -   The variable numLeftBelow is set equal to 0 and availLB is             set equal to TRUE.         -   When predModeIntra is equal to INTRA_L_CCLM, the following             applies for y=nTbH . . . 2*nTbH−1 until availLB is equal to             FA LSE or y is equal to 2*nTbH−1:             -   The availability derivation process for a block as                 specified in clause 6.4.X [Ed. (BB): Neighbouring blocks                 availability checking process tbd] is invoked with the                 current chroma location (xCurr, yCurr) set equal to                 (xTbC, yTbC) and the neighbouring chroma location                 (xTbC−1, yTbC+y) as inputs, and the output is assigned                 to availableLB             -   When availableLB is equal to TRUE, numLeftBelow is                 incremented by one.                 The number of available neighbouring chroma samples on                 the top and top-right numTopSamp and the number of                 available neighbouring chroma samples on the left and                 left-below nLeftSamp are derived as follows:     -   If predModeIntra is equal to INTRA_LT_CCLM, the following         applies:

numSampT=availT?nTbW:0  (8-157)

numSampL=availL?nTbH:0  (8-158)

-   -   Otherwise, the following applies:

numSampT=(availT&& predModeIntra==INTRA_T_CCLM)?(nThW+Min(numTopRight,nTbH)):0  (8-159)

numSampL=(availL&& predModeIntra==INTRA_L_CCLM)?(nTbH+Min(numLeftBelow,nTbW)):0  (8-160)

The variable bCTUboundary is derived as follows:

bCTUboundary=(yTbC & (<<(Ctb Log 2SizeY−1)−1)==0)?TRUE:FALSE.   (8-161)

The variable cntN and array pickPosN[ ] with N being replaced by L and T, are derived as follows:

-   -   The variable numIs4N is set equal to ((availT && availL &&         predModentra==INTRA_LT_CCLM) ? 0:1).     -   The variable startPosN is set equal to numSampN>>(2+numIs4N).     -   The variable pickStepN is set equal to Max(1,         numSampN>>(1+numIs4N).     -   If availN is equal to TRUE and predModeIntra is equal to         INTRA_LT_CCLM or INTRA N CCLM, cntN is set equal to         Min(numSampN. (1+numIs4N)<<1), and pickPosN[pos] is set equal to         (startPosN+pos*pickStepN), with pos=0 . . . (cntN−1).     -   Otherwise, cntN is set equal to 0.         The prediction samples predSamples[x][y] with x=0 . . . nTW−1,         y=0 . . . nTbH−1 are derived as follows:     -   If both numSampL and numSampT are equal to 0, the following         applies:

predSamples[x][y]=1<<(BitDepth_(C)−1)  (8-162)

-   -   Otherwise, the following ordered steps apply:         -   1. The collocated luma samples pY[x][y] with x=0 . . .             nTbW*2−1, y=0 . . . nTbH*2−1 are set equal to the             reconstructed luma samples prior to the deblocking filter             process at the locations (xTbY+x, yTbY+y).         -   2. The neighbouring luma samples pY[x][y] are derived as             follows:             -   When numSampL is greater than 0, the neighbouring left                 luma samples pY[x][y] with x=−1 . . . −3, y=0 . . .                 2*mumSampL−1, are set equal to the reconstructed luma                 samples prior to the deblocking filter process at the                 locations (xTbY+x, yTbY+y).             -   When numSampT is greater than 0, the neighbouring top                 luma samples pY[x][y] with x=0 . . . 2*numSampT−1, y=−1,                 −2, are set equal to the reconstructed luma samples                 prior to the deblocking filter process at the locations                 (xTbY+x, yTbY+y).             -   When availTL is equal to TRUE, the neighbouring top-left                 luma samples pY[x][y] with x=−1, y=−1, −2, are set equal                 to the reconstructed luma samples prior to the                 deblocking filter process at the locations (xTbY+x,                 yTbY+y).         -   3. The down-sampled collocated luma samples pDsY[x][y] with             x=0 . . . nTbW−1, y=0 . . . nTbH−1 are derived as follows:             -   If sys_cclm_colocated_chroma_flag is equal to 1, the                 following applies:                 -   pDsY[x][y] with x=1 . . . nTbW−1, y=1 . . . nTbH−1                     is derived as follows:

pDsY[x][y]=(pY[2*x][2*y−1]+pY[2*x−1][2*y]+4*pY[2*x][2*y]+pY[2*x+1][2*y]+pY[2*x][2*y+1]30 4)>>3  (8-163)

-   -   -   -   -   If availL is equal to TRUE, pDsY[0][y] with y=1 . .                     . nTbH−1 is derived as follows:

pDsY[0][y]=(pY[0][2*y−1]+pY[−1][2*y+]+4*pY[0][2*y]+pY[1][2*y]+pY[0][2*y+1]+4)>>3  (8-164)

-   -   -   -   -   Otherwise, pDsY[0][y] with y=1 . . . nTbH−1 is                     derived as follows:

pDsY[0][y]=(pY[0][2*y−1]+2*pY[0][2*y]+pY[0][2*y+1]+2)>>2  (8-165)

-   -   -   -   -   If availT is equal to TRUE, pDsY[x][0] with x=1 . .                     . nTbW−1 is derived as follows:

pDsY[x][0]=(pY[2*x][−1]+pY[2*x−1][0]+4*pY[2*x][0]+pY[2*x+1][0]+pY[2*x][1]+4)>>3  (8-166)

-   -   -   -   -   Otherwise, pDsY[x][0] with x=1 . . . nTbW−1 is                     derived as follows:

pDsY[x][0]=(pY[2*x−1][0]+2*pY[2*x][0]+pY[2*x+1][0]+2)>>2  (8-167)

-   -   -   -   -   If availL is equal to TRUE and availT is equal to                     TRUE, pDsY[0][0] is derived as follows:

pDsY[0][0]=(pY[0][−1]+pY[−1][0]+4*pY[0][0]+pY[1][0]+pY[0][1]+4)>>3  (8-168)

-   -   -   -   -   Otherwise if availL is equal to TRUE and availT is                     equal to FALSE, pDsY[0][0] is derived as follows:

pDsY[0][0]=(pY[−1][0]+2*pY[0][0]+pY[1]0]+2)>>2   (8-169)

-   -   -   -   -   Otherwise if availL is equal to FALSE and availT is                     equal to TRUE, pDsY[0][0] is derived as follows:

pDsY[0][0]=(pY[0][−1]+2*pY[0][0]+pY[0][1]+2)>>2   (8-170)

-   -   -   -   -   Otherwise (availL is equal to FALSE and availT is                     equal to FALSE), pDsY[0][0] is derived as follows:

pDsY[0][0]=pY[0][0]  (8-171)

-   -   -   -   -   Otherwise, the following applies:                 -    pDsY[x][y] with x=1 . . . nTbW−1, y=0 . . . nTbH−1                     is derived as follows:

pDsY[x][y]=(pY[2*x−1][2*y]+pY[2*x−1][2*y+1]+2*pY[2*x][2*y]+2*pY[2*x][2*y+1]+pY[2*x+1][2*y]+pY[2*x+1][2*y+1]+4)>>3  (8-172)

-   -   -   -   -    If availL is equal to TRUE, pDsY[0][y] with y=0 . .                     . nTbH−1 is derived as follows:

pDsY[0][y]=(pY[−1][2*y]+pY[−1][2*y+1]+2*pY[0][2*y]+2*pY[0][2*y+1]+pY[1][2*y]+pY[1][2*y+1]+4)>>3  (8-173)

-   -   -   -   -    Otherwise, pDsY[0][y] with y=0 . . . nTbH−1 is                     derived as follows:

pDsY[0][y]=(pY[0][2*y]+pY[0][2*y+1]+1)>>1  (8-174)

-   -   -   4. When numSampL is greater than 0, the selected             neighbouring left chroma samples pSelC[idx] are set equal to             p[−1][pickPosL[idx]] with idx=0 . . . (cntL−1), and the             selected down-sampled neighbouring left luma samples             pSelDsY[idx] with idx=0 . . . (cntL−1) are derived as             follows:             -   The variable v is set equal to pickPosL[idx].             -   If sps_cclm_colocated_chroma_flag is equal to 1, the                 following applies:                 -   If y>0∥availTL==TRUE,

pSelDsY[idx]=(pY[−2][2*y−1]+pY[−3][2*y]+4*pY[−2][2*y]+pY[−1][2*y]+pY[−2][2*y+1]+4)>>3  (8-175)

-   -   -   -   -   Otherwise,

pSelDsY[idx]=(pY[−3][0]+2*pY[−2][0]+pY[−1][0]+2)>>2   (8-177)

-   -   -   -   Otherwise, the following applies:

pSelDsY[idx]=(pY[−1][2*y]+pY[−1][2*y−1]+2*pY[−2][2*y]+2*pY[−2][2*y+1]+pY[−3][2*y]+pY[−3][2*y+1]+4)>>3  (8-178)

-   -   -   5. When numSampT is greater than 0, the selected             neighbouring top chroma samples pSelC[idx] are set equal to             p[pickPosT[idx−cntL]][−1] with idx=cntL . . . (cntL+cntT−1)             and the down-sampled neighbouring top luma samples             pSelDsY[idx] with idx=cntL . . . (cntL+cntT−1) are specified             as follows:             -   The variable x is set equal to pickPosT[idx−cntL].             -   If sps_cclm_colocated_chroma_flag is equal to 1, the                 following applies:                 -   If x>0:                 -    If bCTUboundary is equal to FALSE, the following                     applies:

pSelDsY[idx]=(pY[2*x][−3]+pY[2*x−1][−2]+4*pY[2*x][−2]+pY[2*x+1][−2]+pY[2*x][−1]+4)>>3  (8-179)

-   -   -   -   -    Otherwise (bCTUboundary is equal to TRUE), the                     following applies:

pSelDsY[idx]=(pY[2*x−1][−1]+2*pY[2*x][−1]+pY[2*x+1][−1]+2)>>2  (8-180)

-   -   -   -   -   Otherwise:                 -    If availTL is equal to TRUE and bCTUboundary is                     equal to FALSE, the following applies:

pSelDsY[idx]=(pY[0][−3]+pY[−1][−2]+4*pY[0][−2]+pY[1][−2]+pY[0][−1]+4)>>3  (8-181)

-   -   -   -   -    Otherwise if availTL is equal to TRUE and                     bCTUboundary is equal to TRUE, the following                     applies:

pSelDsY[idx]=(pY[−1][−1]+2*pY[0][−1]+pY[1][−]+2)>>2  (8-182)

-   -   -   -   -    Otherwise if availTL is equal to FALSE and                     bCTUboundary is equal to FALSE, the following                     applies:

pSelDsY[idx]=(pY[0][−3]+2*pY[0][−2]+pY[0][−1]+2)>>2  (8-183)

-   -   -   -   -    Otherwise (availTL is equal to FALSE and                     bCTUboundary is equal to TRUE), the following                     applies:

pSelDsY[idx]=pY[0][−1]   (8-184)

-   -   -   -   Otherwise, the following applies:                 -   If x>0:                 -    If bCTUboundary is equal to FALSE, the following                     applies:

pSelDsY[idx]=(pY[2*x−1][−2]+pY[2*x−1][−1]+2*pY[2*x][−2]+2*pY[2*x][−1]+pY[2*x+1][−2]+pY[2*x+1][−1]+4)>>3  (8-185)

-   -   -   -   -    Otherwise (bCTUboundary is equal to TRUE), the                     following applies:

pSelDsY[idx](pY[2*x−1][−1]+2*pY[2*x][−1]+pY[2*x+1][−1]+2)>>2  (8-186)

-   -   -   -   Otherwise:                 -   If availTL is equal to TRUE and bCTUboundary is                     equal to FALSE, the following applies:

pSelDsY[idx]=(pY[−1][−2]+pY[−1][−]+2*pY[0][−2]+2*pY[0][−1]+pY[1][−2]+pY[1][−1]+4)>>3  (8-187)

-   -   -   -   -   Otherwise if availTL is equal to TRUE and                     bCTUboundary is equal to TRUE, the following                     applies:

pSelDsY[idx]=(pY[−1][−1]+2*pY[0][−1]+pY[I][−1]+2)>>2  (8-188)

-   -   -   -   -   Otherwise if availTL is equal to FALSE and                     bCTUboundary is equal to FALSE, the following                     applies:

pSelDsY[idx](pY[0][−2]+pY[0][−1]+1)>>1  (8-189)

-   -   Otherwise (availTL is equal to FALSE and bCTUboundary is equal         to TRUE), the following applies:

pSelDsY[idx]=pY[0][−1]   (8-190)

-   -   -   6. When cntT+cntL is not equal to 0, the variables minY,             maxY, minC and maxC are derived as follows:             -   When cntT+cntL is equal to 2, set pSelComp[3] equal to                 pSelComp[0], pSelComp[2] equal to pSelComp[1],                 pSelComp[0] equal to pSelComp[1], and pSelComp[1] equal                 to pSelComp[3], with Comp being replaced by DsY and C.             -   The arrays minGrpIdx[ ] and maxGrpIdx[ ] are set as:                 minGrpIdx[0]=0, minGrpIdx[1]=2, maxGrpIdx[0]=1,                 maxGrpIdx[1]=3.             -   If pSelDsY[minGrpIdx[0]]>pSelDsY[minGrpIdx[1]],                 Swap(minGrpIdx[0], minGrpIdx[1]).             -   If pSelDsY[maxGrpIdx[0]]>pSelDsY[maxGrpIdx[1]],                 Swap(maxGrpIdx[0], maxGrpIdx[1]).             -   If pSelDsY[minGrpIdx[1]]>pSelDsY[maxGrpIdx[1]],                 Swap(minGrpIdx[1], maxGrpIdx[0]).             -   If pSelDsY[minGrpIdx[1]]>pSelDsY[maxGrpIdx[0]],                 Swap(minGrpIdx[1], maxGrpIdx[0]).             -   maxY=(pSelDsY[maxGrpIdx[0]]+pSelDsY[maxGrpIdx[1]]+1)>>1.             -   maxC=(pSelC[maxGrpIdx[0]]+pSelC[maxGrpIdx[1]]+1)>>1.             -   minY=(pSelDsY[minGrpIdx[0]]+pSelDsY[minGrpIdx[1]]+1)>>1.             -   minC=(pSelC[minGrpIdx[0]]+pSelC[minGrpIdx[1]]+1)>>1.         -   7. The variables a, b, and k are derived as follows:             -   If numSampL is equal to 0, and numSampT is equal to 0,                 the following applies:

k=0  (8-208)

a=0  (8-209)

b=1<<(BitDepth_(C)−1)  (8-210)

-   -   -   -   Otherwise, the following applies:

diff=maxY−minY  (8-211)

-   -   -   -   -   If diff is not equal to 0, the following applies:

diffC=maxC−minC  (8-212)

x=Floor(Log 2(diff))  (8-213)

normDiff=((diff<<4)>>x) & 15  (8-214)

x+=(normDiff !=0)?1:0  (8-215)

y=Floor(Log 2(Abs(diffC)))+1  (8-216)

a=(diffC*(divSigTable[normDiff]|8)+2^(y−1))>>y  (8-217)

k=((3+x−y)<1)?1:3+x−y  (8-218)

a=((3+x−y)<1)?Sign(a)*15:a  (8-219)

b=minC−((a*minY)>>k)  (8-220)

-   -   -   -   -   where divSigTable[ ] is specified as follows:

divSigTable[ ]={0,7,6,5,5,4,4,3,3,2,2,1,1,1,1,0}  (8-221)

-   -   -   -   -   Otherwise (diff is equal to 0), the following                     applies:

k=0  (8-222)

a=0  (8-223)

b=minC  (8-224)

-   -   -   8. The prediction samples predSamples[x][y] with x=0 . . .             nTbW−1, y=0 . . . nTbH−1 are derived as follows:

predSamples[x][y]=Clip1C(((pDsY[x][y]*a)>>k)+b)  (8-225)

2.2.3.3 Miscellaneous Intra Prediction Aspects

VTM4 includes many intra coding tools which are different from HEVC, for example, the following features have been included in the VVC test model 3 on top of the bock tree structure.

-   -   67 intra mode with wide angles mode extension     -   Block size and mode dependent 4 tap interpolation filter     -   Position dependent intra prediction combination (PDPC)     -   Cross component linear model intra prediction     -   Multi-reference line intra prediction     -   Intra sub-partitions

2.2.4 Inter Prediction in VVC 2.2.4.1 Combined Inter and Intra Prediction (CUP)

In VTM4, when a CU is coded in merge mode, and if the CU contains at least 64 luma samples (that is, CU width times CU height is equal to or larger than 64), an additional flag is signaled to indicate if the combined inter/intra prediction (CIIP) mode is applied to the current CU.

In order to form the CIIP prediction, an intra prediction mode is first derived from two additional syntax elements. Up to four possible intra prediction modes can be used: DC, planar, horizontal, or vertical. Then, the inter prediction and intra prediction signals are derived using regular intra and inter decoding processes. Finally, weighted averaging of the inter and intra prediction signals is performed to obtain the CIIP prediction.

2.2.4.2 Miscellaneous Inter Prediction Aspects

VTM4 includes many inter coding tools which are different from HEVC, for example, the following features have been included in the VVC test model 3 on top of the bock tree structure.

-   -   Affine motion inter prediction     -   Sub-block based temporal motion vector prediction     -   Adaptive motion vector resolution     -   8×8 block based motion compression for temporal motion         prediction     -   High precision ( 1/16 pel) motion vector storage and motion         compensation with 8-tap interpolation filter for luma component         and 4-tap interpolation filter for chroma component     -   Triangular partitions     -   Combined intra and inter prediction     -   Merge with MVD (MMVD)     -   Symmetrical MVD coding     -   Bi-directional optical flow     -   Decoder side motion vector refinement     -   Bi-predictive weighted averaging

2.2.5 In-loop filters

There are totally three in-loop filters in VTM4. Besides deblocking filter and sample adaptive offset (SAO) (the two loop filters in HEVC), adaptive loop filter (ALF) are applied in the VTM4. The order of the filtering process in the VTM4 is the deblocking filter, SAO and ALF.

In the VTM4, the SAO and deblocking filtering processes are almost the same as those in HEVC.

In the VTM4, a new process called the luma mapping with chroma scaling was added (this process was previously known as the adaptive in-loop reshaper). This new process is performed before deblocking.

2.2.6 Luma Mapping with Chroma Scaling (LMCS, aka. In-Loop Reshaping)

In VTM4, a coding tool called the luma mapping with chroma scaling (LMCS) is added as a new processing block before the loop filters. LMCS has two main components: 1) in-loop mapping of the luma component based on adaptive piecewise linear models; 2) for the chroma components, luma-dependent chroma residual scaling is applied. FIG. 4 shows the LMCS architecture from decoder's perspective. The light-blue shaded blocks in FIG. 4 indicate where the processing is applied in the mapped domain; and these include the inverse quantization, inverse transform, luma intra prediction and adding of the luma prediction together with the luma residual. The unshaded blocks in FIG. 4 indicate where the processing is applied in the original (i.e., non-mapped) domain; and these include loop filters such as deblocking, ALF, and SAO, motion compensated prediction, chroma intra prediction, adding of the chroma prediction together with the chroma residual, and storage of decoded pictures as reference pictures. The light-yellow shaded blocks in FIG. 4 are the new LMCS functional blocks, including forward and inverse mapping of the luma signal and a luma-dependent chroma scaling process. Like most other tools in VVC, LMCS can be enabled/disabled at the sequence level using a SPS flag.

2.2.6.1 Luma Mapping with Piecewise Linear Model

The in-loop mapping of the luma component adjusts the dynamic range of the input signal by redistributing the codewords across the dynamic range to improve compression efficiency. Luma mapping makes use of a forward mapping function, FwdMap, and a corresponding inverse mapping function, InvMap. The FwdMap function is signaled using a piecewise linear model with 16 equal pieces. InvMap function does not need to be signaled and is instead derived from the FwdMap function.

The luma mapping model is signaled at the tile group level. A presence flag is signaled first. If luma mapping model is present in the current tile group, corresponding piecewise linear model parameters are signaled. The piecewise linear model partitions the input signal's dynamic range into 16 equal pieces, and for each piece, its linear mapping parameters are expressed using the number of codewords assigned to that piece. Take 10-bit input as an example. Each of the 16 pieces will have 64 codewords assigned to it by default. The signaled number of codewords is used to calculate the scaling factor and adjust the mapping function accordingly for that piece. At the tile group level, another LMCS enable flag is signaled to indicate if the LMCS process as depicted in FIG. 4 is applied to the current tile group.

Each i-th piece, i=0 . . . 15, of the FwdMap piecewise linear model is defined by two input pivot points InputPivot[ ] and two output (mapped) pivot points MappedPivot[ ].

The InputPivot[ ] and MappedPivot[ ] are computed as follows (assuming 10-bit video):

-   -   1) OrgCW=64     -   2) For i=0:16, InputPivot[i]=i*OrgCW     -   3) For i=0:16, MappedPivot[i] is calculated as follows:         -   MappedPivot[0]=0;         -   for(i=0; i<16; i++)             -   MappedPivot[i+1]=MappedPivot[i]+SignalledCW[i]                 where SignalledCW[i] is the signaled number of codewords                 for the i-th piece.

As shown in FIG. 4 , for an inter-coded block, motion compensated prediction is performed in the mapped domain. In other words, after the motion-compensated prediction block Y_(pred) is calculated based on the reference signals in the DPB, the FwdMap function is applied to map the luma prediction block in the original domain to the mapped domain, Y′_(pred)=FwdMap(Y_(pred)). For an intra-coded block, the FwdMap function is not applied because intra prediction is performed in the mapped domain. After reconstructed block Y_(r) is calculated, the InvMap function is applied to convert the reconstructed luma values in the mapped domain back to the reconstructed luma values in the original domain (Ŷ_(i)=InvMap(Y_(r))). The InvMap function is applied to both intra- and inter-coded luma blocks.

The luma mapping process (forward and/or inverse mapping) can be implemented using either look-up-tables (LUTs) or using on-the-fly computation. If LUT is used, then FwdMapLUT and InvMapLUT can be pre-calculated and pre-stored for use at the tile group level, and forward and inverse mapping can be simply implemented as FwdMap(Y_(pred))=FwdMapLUT[Y_(pred)] and InvMap(Y_(r))=InvMapLUT[Y_(r)], respectively. Alternatively, on-the-fly computation may be used. Take forward mapping function FwdMap as an example. In order to figure out the piece to which a luma sample belongs, the sample value is right shifted by 6 bits (which corresponds to 16 equal pieces). Then, the linear model parameters for that piece are retrieved and applied on-the-fly to compute the mapped luma value. Let i be the piece index, a1, a2 be InputPivot[i] and InputPivot[i+1], respectively, and b1, b2 be MappedPivot[i] and MappedPivot[i+1], respectively. The FwdMap function is evaluated as follows:

FwdMap(Y _(pred))=((b2−b1)/(a2−a1))*(Y _(pred) −a1)+b1

The InvMap function can be computed on-the-fly in a similar manner, except that conditional checks need to be applied instead of a simple right bit-shift when figuring out the piece to which the sample value belongs, because the pieces in the mapped domain are not equal sized.

2.2.6.2 Luma-Dependent Chroma Residual Scaling

Chroma residual scaling is designed to compensate for the interaction between the luma signal and its corresponding chroma signals. Whether chroma residual scaling is enabled or not is also signaled at the tile group level. If luma mapping is enabled and if dual tree partition (also known as separate chroma tree) is not applied to the current tile group, an additional flag is signaled to indicate if luma-dependent chroma residual scaling is enabled or not. When luma mapping is not used, or when dual tree partition is used in the current tile group, luma-dependent chroma residual scaling is disabled. Further, luma-dependent chroma residual scaling is always disabled for the chroma blocks whose area is less than or equal to 4.

Chroma residual scaling depends on the average value of the corresponding luma prediction block (for both intra- and inter-coded blocks). Denote avgY′ as the average of the luma prediction block. The value of C_(ScaleInv) is computed in the following steps:

-   -   1) Find the index Y_(Idx) of the piecewise linear model to which         avgY′ belongs based on the InvMap function.     -   2) C_(ScaleInv)=cScaleInv[Y_(Idx)], where cScaleInv[ ] is a         pre-computed 16-piece LUT.         If the current block is coded as intra, combined inter and intra         prediction (CIIP), or intra block copy ((IBC), a.k.a. current         picture referencing (CPR)) modes, avgY′ is computed as the         average of the intra-, CIIP-, or IBC-predicted luma values;         otherwise, avgY′ is computed as the average of the forward         mapped inter predicted luma values (Y′_(pred) in FIG. 4 ).         Unlike luma mapping, which is performed on the sample basis,         C_(scaleInv) is a constant value for the entire chroma block.         With C_(ScaleInv), chroma residual scaling is applied as         follows:     -   Encoder side:         C_(ResScale)=C_(Res)*C_(Scale)=C_(Res)/C_(ScaleInv)     -   Decoder side:         C_(Res)=C_(ResScale)/C_(Scale)=C_(ResScale)*C_(ScaleInv)

2.2.6.3 Corresponding Working Draft in JVET-M1001_v7 with the Adoption in JVET-N0220

The following spec is based on the modified working draft of JVET G-M1001 and the adoption in JVET-N0220. The modification in the adopted JVET-N0220 is shown in bold and underlining.

Syntax Tables In 7.3.2.1 Sequence Parameter Set RBSP Syntax

 

 

   

   

 

   

   

 

   

 

In 7.3.4.1 General Tile Group Header Syntax

 

 

   

 

   

   

 

   

   

 

   

   

 

   

 

 

   

 

   

   

  if(NumTilesInCurrTileGroup > 1 ) {  

  ue(v) for( i = 0; i < NumTilesInCurrTileGroup - 1; i+ + )  

 

  u(v) } In 7.3.4.4 Luma Mapping with Chroma Scaling Data Syntax

 

   

   

   

   

   

   

   

   

   

 

   

   

   

  -  

   

   

   

 

Semantics In 7.4.3.1 Sequence Parameter Set RBSP Semantics

sps_lmcs_enabled_flag equal to 1 specifies that luma mapping with chroma scaling is used in the CVS. sps_lmcs_enabled_flag equal to 0 specifies that luma mapping with chroma scaling is not used in the CVS. tile_group_lmcs_model_present_flag equal to 1 specifies that lmcs_data( ) is present in the tile group header. tile_group_lmcs_model_present_flag equal to 0 specifies that lmcs_data( ) is not present in the tile group header. When tile_group_lmcs_model_present_flag is not present, it is inferred to be equal to 0. tile_group_lmcs_enabled_flag equal to 1 specifies that luma mapping with chroma scaling is enabled for the current tile group. tile_group_lmcs_enabled_flag equal to 0 specifies that luma mapping with chroma scaling is not enabled for the current tile group. When tile_group_lmcs_enabled_flag is not present, it is inferred to be equal to 0. tile_group_chroma_residual_scale_flag equal to 1 specifies that chroma residual scaling is enabled for the current tile group. tile_group_chroma_residual_scale_flag equal to 0 specifies that chroma residual scaling is not enabled for the current tile group. When tile group chroma_residual_scale_flag is not present, it is inferred to be equal to 0. In 7.4.5.4 Luma Mapping with Chroma Scaling Data Semantics lmcs_min_bin_idx specifies the minimum bin index used in the luma mapping with chroma scaling construction process. The value of lmcs_min_bin_idx may be in the range of 0 to 15, inclusive. lmcs_delta_max_bin_idx specifies the delta value between 15 and the maximum bin index LmcsMaxBinIdx used in the luma mapping with chroma scaling construction process. The value of lmcs_delta_max_bin_idx may be in the range of 0 to 15, inclusive. The value of LmcsMaxBinIdx is set equal to 15−lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx may be larger than or equal to lmcs_min_bin_idx. lmcs_delta_cw_prec_minus1 plus 1 specifies the number of bits used for the representation of the syntax lmcs_delta_abs_cw[i]. The value of lmcs_delta_cw_prec_minus1 may be in the range of 0 to BitDepthY−2, inclusive. lmcs_delta_abs_cw[i] specifies the absolute delta codeword value for the ith bin. lmcs_delta_sign_cw_flag[i] specifies the sign of the variable lmcsDeltaCW[i] as follows:

-   -   If lmcs_delta_sign_cw_flag[i] is equal to 0, lmcsDeltaCW[i] is a         positive value.     -   Otherwise (lmcs_delta_sign_cw_flag[i] is not equal to 0),         lmcsDeltaCW[i] is a negative value.         When lmcs_delta_sign_cw_flag[i] is not present, it is inferred         to be equal to 0.         The variable OrgCW is derived as follows:

OrgCW=(1<<BitDepth_(Y))/16  (7-70)

The variable lmcsDeltaCW[i], with i=lmcs_min_bin_idx . . . LmcsMaxBinIdx, is derived as follows:

lmcsDeltaCW[i]=(1−2*lmcs_delta_sign_cw_flag[i])*lmcs_delta_abs_cw[i]  (7-71)

The variable lmcsCW[i] is derived as follows:

-   -   For i=0 . . . lmcs_min_bin_idx 1, lmcsCW[i] is set equal 0.     -   For i=lmcs_min_bin_idx . . . LmcsMaxBinIdx, the following         applies:

lmcsCW[i]=OrgCW+lmcsDeltaCW[i]  (7-72)

-   -   The value of lmcsCW[i] shall be in the range of (OrgCW>>3) to         (OrgCW<<3−1), inclusive.     -   For i=LmcsMaxBinIdx+1 . . . 15, lmcsCW[i] is set equal 0.         In bitstream conformance, the following condition may be true:

Σ_(i=0) ¹⁵lmcsCW[i]<=(1<<BitDepth_(Y))−1  (7-73)

The variable InputPivot[i], with i=0 . . . 16, is derived as follows:

InputPivot[i=i]*OrgCW  (7-74)

The variable LmcsPivot[i] with i=0 . . . 16, the variables ScaleCoeff[i] and InvScaleCoeff[i] with i=0 . . . 15, are derived as follows:

 LmcsPivot[ 0 ] = 0;  for( i = 0; i <= 15; i++ ) {  LmcsPivot[ i + 1 ] = LmcsPivot[ i ] + lmcsCW[ i ]  

 

 

 

 

   if ( lmcsCW[ i ] = = 0 )     InvScaleCoeff[ i ] = 0      else

 

 } The variable ChromaScaleCoeff[ i ], with i = 0...15, is derived as follows:  if ( lmcsCW[ i ] = = 0 )   ChromaScaleCoeff[ i ] = (1 << 11)  else {

 

 } The variables ClipRange, LmcsMinVal, and LmcsMaxVal are derived as follows:

ClipRange=((lmcs_min_bin_idx>0)&&(LmcsMaxBinIdx<15)  (7-77)

LmcsMinVal=16<<(BitDepth_(Y)−8)  (7-78)

LmcsMaxVal=235<<(BitDepth_(Y)−8)  (7-79)

-   -   NOTE Arrays InputPivot[i] and LmcsPivot[i], ScaleCoeff[i], and         InvScaleCoeff[i], ChromaScaleCoeff[i], ClipRange, LmcsMinVal and         LmcsMaxVal, are updated only when         tile_group_lmcs_model_present_flag is equal to 1. Thus, the lmcs         model may be sent with an IRAP picture, for example, but lmcs is         disabled for that IRAP picture.

3. Drawbacks of Existing Implementations

The current design of luma mapping with chroma scaling (LMCS)/cross-component linear model (CCLM) may have the following problems:

-   1. In LMCS coding tool, the chroma residual scaling factor is     derived by the average value of the collocated luma prediction     block, which results in a latency for processing the chroma samples     in LMCS chroma residual scaling.     -   a) In case of single/shared tree, the latency is caused by (a)         waiting for all the prediction samples of the whole luma block         available, and (b) averaging all the luma prediction samples         obtained by (a).     -   b) In case of dual/separate tree, the latency is even worse         since separate block partitioning structure for luma and chroma         components is enabled in I slices. Therefore, one chroma block         may correspond to multiple luma blocks, and one 4×4 chroma block         may correspond to a 64×64 luma block. Thus the worst case is         that the chroma residual scaling factor of current 4×4 chroma         block may need to wait until all the prediction samples in the         whole 64×64 luma block are available. In a word, the latency         issue in dual/separate tree would be much more serious. -   2. In CCLM coding tool, the CCLM model computation for intra chroma     prediction depends on the left and above reference samples of both     luma block and chroma block. And the CCLM prediction for a chroma     block depends on the collocated luma reconstructed samples of the     same CU. This would cause high latency in dual/separate tree.     -   In case of dual/separate tree, one 4×4 chroma block may         correspond to a 64×64 luma block. Thus, the worst case is that         the CCLM process for the current chroma block may need to wait         until the corresponding whole 64×64 luma block being         reconstructed. This latency issue is similar as LMCS chroma         scaling in dual/separate tree.

4. Example Techniques and Embodiments

To tackle the problems, we propose several methods to remove/reduce/restrict the cross-component dependency in luma-dependent chroma residual scaling, CCLM, and other coding tools that rely on information from a different color component.

The detailed embodiments described below should be considered as examples to explain general concepts. These embodiments should not be interpreted narrowly way. Furthermore, these embodiments can be combined in any manner.

It is noted that although the bullets described below explicitly mention LMCS/CCLM, the methods may be also applicable to other coding tools that rely on information from a different color component. In addition, the term ‘luma’ and ‘chroma’ mentioned below may be replaced by ‘a first color component’ and ‘a second color component’ respectively, such as ‘G component’ and ‘B/R component’ in the RGB color format.

In the following discussion, the definition a “collocated sample/block” aligns with the definition of collocated sample/block in VVC working draft JVET-M1001. To be more specific, in 4:2:0 color format, suppose the top-left sample of a chroma block is at position (xTbC, yTbC), then the top-left sample of the collocated luma block location (xTbY, yTbY) is derived as follows: (xTbY, yTbY)=(xTbC<<1, yTbC<<1). As illustrated in FIG. 5 , the top-left sample of the current chroma block is located at (x=16, y=16) in the chroma picture, then the top-left sample of its collocated luma block is located at (x=32, y=32) in the luma picture, regardless of the block partition of collocated luma block in the luma picture. For another example, saying in the same color component, the location of the top-left sample of the collocated block in the reference frame should be the same with the location of the top-left sample of the current block in the current frame, as illustrated in FIG. 6 , suppose the top-left sample of the current block is (x,y) in the current frame, then the top-left sample of the collocated block of the current block have the same location (x,y) in the reference frame.

In the following discussion, a “corresponding block” may have different location with the current block. For an example, there might be a motion shift between the current block and its corresponding block in the reference frame. As illustrated in FIG. 6 , suppose the current block is located at (x,y) in the current frame and it has a motion vector (mv_(x), mv_(y)), then a corresponding block of the current block may be located at (x+mv_(x), y+mv_(y)) in the reference frame. And for an IBC coded block, the collocated luma block (pointed by zero vector) and the corresponding luma block (pointed by non-zero-BV) may locate in different places of the current frame. For another example, when the partition of luma block doesn't align with the partition of chroma block (in dual tree partition of I slices), the collocated luma block of the current chroma block may belong to a larger luma block which depends on the partition size of the overlapped luma coding block covering the top-left sample of the collocated luma block. As illustrated in FIG. 5 , assume the bold rectangle denotes the partitions of the block, so that a 64×64 luma block is firstly split by a BT and then the right part of the 64×64 luma block is further split by a TT, which results in three luma blocks of sizes equal to 32×16, 32×32, 32×16, respectively. Thus, looking at the top-left sample (x=32, y=32) of the collocated luma block of the current chroma block, it belongs to the center 32×32 luma block of the TT partition. In this case, we call the corresponding luma block that covers the top-left sample of the collocated luma block as a “corresponding luma block”. Therefore, in this example, the top-left sample of the corresponding luma block is located at (x=32, y=16).

Hereinafter, decoder-side motion vector derivation (DMVD) is used to represent Bi-Directional Optical Flow (BDOF) (a.k.a BIO) or/and decode-side motion vector refinement (DMVR) or/and frame rate up-conversion (FRUC) or/and other method that refines motion vector or/and prediction sample value at decoder.

Removal of the Chroma Scaling Latency of LMCS and Model Computation of CCLM

-   1. It is proposed that for an inter-coded block, one or multiple     reference samples of the current block in reference frames may be     used to derive the chroma residual scaling factor in the LMCS mode.     -   a) In one example, reference luma samples may be directly used         to derive the chroma residual scaling factor.         -   i. Alternatively, interpolation may be firstly applied to             reference samples and the interpolated samples may be used             to derive the chroma residual scaling factor.         -   ii. Alternatively, reference samples in different reference             frames may be utilized to derive the final reference samples             that are used for the chroma residual scaling factor             derivation.             -   1) In one example, for bi-prediction coded blocks, the                 above method may be applied.         -   iii. In one example, the intensities of reference samples             may be converted to reshaping domain before being used to             derive the chroma residual scaling factor.         -   iv. In one example, linear combination of the reference             samples may be used to derive the chroma residual scaling             factor.             -   1) For example, a×S+b may be used to derive the chroma                 residual scaling factor, where S is a reference sample,                 a and b are parameters. In one example, a and b may be                 derived by Localized Illuminate Compensation (LIC).     -   b) In one example, the location of the reference luma samples in         the reference frame may depend on the current block's motion         vector(s).         -   i. In one example, a reference sample belongs to a reference             luma block, which is in a reference picture, and with the             same width and height as the current luma block. The             position of the reference luma sample in the reference             picture may be calculated as the position of its             corresponding luma sample in the current picture, adding a             motion vector.         -   ii. In one example, the position of the reference luma             samples may be derived by the position of top-left (or             center, or bottom-right) sample of the current luma block             and current block's motion vector, referred as a             corresponding luma sample in the reference frame.             -   1) In one example, an integer motion vector may be used                 to derive the corresponding luma sample in the reference                 frame. In one example, the motion vector associated with                 one block may be either rounded toward zero, or rounded                 away from zero to derive the integer motion vector.             -   2) Alternatively, a fractional motion vector may be used                 to derive the corresponding luma sample in the reference                 frame, so that the interpolation process may be used to                 derive the fractional reference samples.         -   iii. Alternatively, the position of the reference luma             samples may be derived by the position of top-left (or             center, or bottom-right) sample of current luma block.         -   iv. Alternatively, multiple corresponding luma samples at             some pre-defined positions in the reference frame may be             picked to calculate the chroma residual scaling factor.     -   c) In one example, the median or average value of the multiple         reference luma samples may be used to derive the chroma residual         scaling factor.     -   d) In one example, the reference luma samples in pre-defined         reference frames may be used to derive the chroma residual         scaling factor.         -   i. In one example, the pre-defined reference frame may be             the one with reference index equal to 0 of reference picture             list 0.         -   ii. Alternatively, the reference index and/or reference             picture list for the pre-defined reference frame may be             signaled in sequence/picture/tile group/slice/tile/CTU             row/video unit level.         -   iii. Alternatively, the reference luma samples in multiple             reference frames may be derived and the averaged or weighted             average values may be utilized to get the chroma residual             scaling factor. -   2. It is proposed that whether and how to derive the chroma residual     scaling factor from luma samples in the LMCS mode may depend on     whether the current block applies bi-prediction.     -   a) In one example, the chroma residual scaling factor is derived         for each prediction direction individually. -   3. It is proposed that whether and how to derive the chroma residual     scaling factor from luma samples in the LMCS mode may depend on     whether the current block applies sub-block-based prediction.     -   a) In one example, the sub-block-based prediction is affine         prediction.     -   b) In one example, the sub-block-based prediction is Alternative         Temporal Motion Vector Prediction (ATMVP).     -   c) In one example, the chroma residual scaling factor is derived         for each sub-block individually.     -   d) In one example, the chroma residual scaling factor is derived         for the whole block even if it is predicted by sub-blocks.         -   i. In one example, motion vector of one selected sub-block             (e.g., top-left sub-block) may be used to identify the             reference samples of current block as described in bullet 1. -   4. It is proposed that the luma prediction values used to derive the     chroma residual scaling factor may be intermediate luma prediction     value instead of the final luma prediction value.     -   a) In one example, the luma prediction values before the process         of Bi-Directional Optical Flow (BDOF, a.k.a. BIO) may be used to         derive the chroma residual scaling factor.     -   b) In one example, the luma prediction values before the process         of Decoder-side Motion Vector Refinement (DMVR) may be used to         derive the chroma residual scaling factor.     -   c) In one example, the luma prediction values before the process         of LIC may be used to derive the chroma residual scaling factor.     -   d) In one example, the luma prediction values before the process         of Prediction Refinement Optical Flow (PROF) as proposed in         JVET-N0236 may be used to derive the chroma residual scaling         factor. -   5. Intermediate motion vectors may be used to identify the reference     samples.     -   a) In one example, motion vector before the process of BDOF         or/and DMVR or/and other DMVD methods may be used to identify         the reference samples.     -   b) In one example, the motion vector before the process of         Prediction Refinement Optical Flow (PROF) as proposed in         JVET-N0236 may be used to identify the reference samples. -   6. The above methods may be applicable when the current block is     coded with inter mode. -   7. It is proposed that for an IBC-coded block, one or multiple     reference samples in reference block of current frame may be used to     derive the chroma residual scaling factor in the LMCS mode. When the     block IBC-coded, the term “motion vector” may also be referred as     “block vector”, where the reference picture is set as the current     picture.     -   a) In one example, a reference sample belongs to a reference         block, which is in the current picture, and with the same width         and height as the current block. The position of the reference         sample may be calculated as the position of its corresponding         sample adding a motion vector.     -   b) In one example, the position of the reference luma samples         may be derived by the position of top-left (or center, or         bottom-right) sample of current luma block adding a motion         vector.     -   c) Alternatively, the position of the reference luma samples may         be derived by the position of top-left (or center, or         bottom-right) sample of current luma block adding current         block's block vector.     -   d) Alternatively, multiple corresponding luma samples at some         pre-defined positions in the reference region of current luma         block may be picked to calculate the chroma residual scaling         factor.     -   e) In one example, multiple corresponding luma samples may be         computed with a function to derive the chroma residual scaling         factor.         -   i. For example, the median or average value of multiple             corresponding luma samples may be computed to derive the             chroma residual scaling factor.     -   f) In one example, the intensities of reference samples may be         converted to reshaping domain before being used to derive the         chroma residual scaling factor.         -   i. Alternatively, the intensities of reference samples may             be converted to original domain before being used to derive             the chroma residual scaling factor. -   8. It is proposed that one or multiple prediction/reconstructed     samples which are located at the identified location(s) of the     current luma block in the current frame may be used to derive the     chroma residual scaling factor for the current chroma block in the     LMCS mode.     -   a) In one example, if current block is inter-coded, the luma         prediction (or reconstruction) sample located in the center of         the current luma block may be picked to derive the chroma         residual scaling factor.     -   b) In one example, the average value of the first M×N luma         prediction (or reconstruction) samples may be picked to derive         the chroma residual scaling factor, where M×N could be smaller         than collocated luma block size width×height. -   9. It is proposed that the whole or partial of the procedure used to     calculate the CCLM model may be used for the chroma residual scaling     factor derivation of current chroma block in the LMCS mode.     -   a) In one example, reference samples which are located at the         identified locations of neighbouring luma samples of the         collocated luma block in CCLM model parameter derivation process         may be utilized to derive chroma residual scaling factor.         -   i. In one example, those reference samples may be directly             used.         -   ii. Alternatively, downsampling may be applied to those             reference samples, and downsampled reference samples may be             applied.     -   b) In one example, K out of S reference samples selected for         CCLM model computation may be used for chroma residual scaling         factor derivation in the LMCS mode. E.g., K is equal to 1 and S         is equal to 4.     -   c) In one example, the average/minimum/maximum value of the         reference samples of the collocated luma block in CCLM mode may         be used for chroma residual scaling factor derivation in the         LMCS mode. -   10. How to select samples for derivation of chroma residual scaling     factors may be dependent on the coded information of current block.     -   a) The coded information may include quantization parameter         (QP), coding mode, picture order count (POC), intra-prediction         mode, motion information and so on.     -   b) In one example, for IBC coded or Non-IBC coded blocks, the         way to select samples may be different.     -   c) In one example, the way to select samples may be different         based on the reference picture information, such as POC distance         between reference pictures and current picture. -   11. It is proposed that the chroma residual scaling factor and/or     the model computation of CCLM may depend on neighboring samples of a     corresponding luma block which covers the top-left sample of the     collocated luma block. In this disclosure, a “coding block” may     refer to a video coding region such as CU/TU/PU as specified in HEVC     specification or in the VVC working draft.     -   a) The “corresponding luma coding block” may be defined as the         coding block which covers the top-left position of the         collocated luma coding block.         -   i. FIG. 5 shows an example, where for an intra-coded chroma             block in dual tree case, the CTU partition of chroma             component may be different from the CTU partition of luma             component. Firstly, a “corresponding luma coding block”             covering the top-left sample of the collocated luma block of             current chroma block is retrieved. Then by using the block             size information of the “corresponding luma coding block”,             the top-left sample of the “corresponding luma coding block”             can be derived, the top-left luma sample of the             “corresponding luma coding block” covering the top-left             sample of the collocated luma block is located at (x=32,             y=16).     -   b) In one example, the block         size/partition/location/coordination may be used to derive the         location of “corresponding luma coding block” that covers the         top-left sample of the collocated luma coding block.         -   i. In one example, the block size, and/or block partition,             and/or block coordination may be stored for each block of a             specific color component, such as the luma component.         -   ii. In one example, “corresponding luma coding block” and             current block may be always inside the same CTU or CTU row,             thus there may be no storage of block             size/partition/position/coordination in the line buffer.     -   c) In one example, the reconstructed samples not in the         “corresponding luma coding block” may be used to derive the         chroma residual scaling factor and/or model computation of CCLM.         -   i. In one example, the reconstructed samples adjacent to the             “corresponding luma coding block” may be used to derive the             chroma residual scaling factor and/or model computation of             CCLM.             -   1) In one example, N samples located at the left                 neighboring columns and/or the above neighboring rows of                 the “corresponding luma coding block” may be used to                 derive the chroma residual scaling factor and/or the                 model computation of CCLM, where N=1 . . . 2 W+2H, W and                 H are the width and height of the “corresponding luma                 coding block”.                 -   a) Suppose the top-left sample of the “corresponding                     luma coding block” is (xCb, yCb), then in one                     example, the above neighboring luma sample may                     locate at (xCb+W/2, yCb−1), or (xCb−1, yCb−1). In an                     alternative example, the left neighboring luma                     sample may locate at (xCb+W−1, yCb−1).                 -   b) In one example, the location(s) of the                     neighboring sample(s) may be fixed, and/or in a                     pre-defined checking order.             -   2) In one example, 1 out of N neighboring samples may be                 selected to derive the chroma residual scaling factor                 and/or the model computation of CCLM. Assume N=3, and                 the checking order of the three neighbor samples (xCb−1,                 yCb−H−1), (xCb+W/2, yCb−1), (xCb−1, yCb−1), then the                 first available neighboring sample in the checking list                 may be selected to derive the chroma residual scaling                 factor.             -   3) In one example, the median or average value of N                 samples located at the left neighboring columns and/or                 the above neighboring rows of the “corresponding luma                 coding block” may be used to derive the chroma residual                 scaling factor and/or the model computation of CCLM,                 where N=1 . . . 2 W+2H, W and H are the width and height                 of the “corresponding luma coding block”.     -   d) In one example, whether to perform the chroma residual         scaling may depend on the “available” neighbouring samples of a         corresponding luma block.         -   i. In one example, the “availability” of neighbouring             samples may depend on the encoding mode of the current             block/sub-block or/and encoding mode of the neighbouring             sample.             -   1) In one example, for a block coded in inter mode,                 neighbouring samples coded in intra mode or/and IBC mode                 or/and CIIP mode or/and LIC mode may be considered as                 “unavailable”.             -   2) In one example, for a block coded in inter mode,                 neighbouring samples employs diffusion filter or/and                 bilateral filter or/and Hadamard transform filter may be                 considered as “unavailable”.         -   ii. In one example, the “availability” of the neighbouring             samples may depend on the width and/or height of the current             picture/tile/tile group/virtual pipeline data unit             (VPDU)/slice.             -   1) In one example, if the neighbouring block locates                 outside the current picture, then it is treated as                 “unavailable”.         -   iii. In one example, when there is no “available”             neighbouring sample, chroma residual scaling may be             disallowed.         -   iv. In one example, when the number of “available”             neighbouring samples is smaller than K (K>=1), chroma             residual scaling may be disallowed.         -   v. Alternatively, the unavailable neighbouring sample may be             filled by a default fixed value, or padding, or             substitution, so that the chroma residual scaling may always             be applied.             -   1) In one example, if the neighbouring sample is not                 available, then it may be filled by 1<<(bitDepth−1),                 where bitDepth specifies the bit depth of the samples of                 the luma/chroma components.             -   2) Alternatively, if the neighbouring sample is not                 available, then it may be filled by padding from the                 surrounding samples located in the left/right/top/bottom                 neighbour.             -   3) Alternatively, if the neighbouring sample is not                 available, then it may be substituted by the first                 available adjacent sample at a pre-defined checking                 order.             -   4) Alternatively, if the neighbouring sample is not                 available, then it may be filled by a predefined                 filtered/mapped value (e.g., filtered/mapped value of                 1<<(bitDepth−1), where bitDepth specifies the bit depth                 of the samples of the luma/chroma components).                 -   a) In one example, the filtering/mapping process may                     be LUT indexing of the forward mapping of LMCS.     -   e) In one example, whether and how to perform the chroma         residual scaling may depend on the coding mode of current block         and/or the coding modes of neighbour blocks.         -   i. The “current block” may refer to the current chroma             block, or it may refer to the collocated luma block, or the             corresponding luma block which covers at least one sample of             the collocated chroma block. The “neighbour blocks”             (adjacent or non-adjacent) may refer to chroma blocks             neighbouring to the current chroma block, or they may refer             to luma blocks neighbouring to the current luma block.         -   ii. In one example, the coding mode of one luma neighbouring             block may be utilized which covers a given position, such as             (−1, −1) relatively to the top-left coordinate of current             block.         -   iii. In one example, the coding modes of multiple             neighbouring blocks may be utilized which cover multiple             positions, such as (x, −1) (e.g., with x being 0, and             block's width minus 1) relatively to the top-left coordinate             of current block, and/or (−1, y) (e.g., with y being −1 . .             . block's height minus 1) relatively to the top-left             coordinate of current block.             -   iv. In one example, if the reconstruction of one                 neighbouring block requires to access samples in the                 current slice/tile group, such as it is X-coded, then                 chroma residual scaling may be disabled.                 -   1) For example, mode X may be intra mode;                 -   2) For example, mode X may be CIIP mode;                 -   3) For example, mode X may be IBC mode;                 -   4) In one example, if current block is inter-coded                     and not CIIP-coded, and the neighbour block                     neighbouring the corresponding luma block is coded                     with mode X, then chroma residual scaling is                     disabled.             -   v. In one example, if the reconstruction of one                 neighbouring block requires to access samples in the                 current slice/tile group, such as it is X-coded, then a                 default value may be used to derive chroma residual                 scaling factor.                 -   1) For example, mode X may be intra mode;                 -   2) For example, mode X may be CIIP mode;                 -   3) For example, mode X may be IBC mode;                 -   4) In one example, if current block is inter-coded                     and not CIIP-coded, and the neighbour block of the                     corresponding luma block is coded with mode X, then                     a default value may be used to derive chroma                     residual scaling factor.                 -   5) In one example, the default value may depend on                     the bit depth of the luma/chroma samples.                 -   6) In one example, the default value may be set to a                     filtered/mapped value of 1<<(bitDepth−1), where                     bitDepth specifies the bit depth of the samples of                     the luma/chroma components. In one example, the                     filtering/mapping process may be a LUT indexing of                     the forward mapping of LMCS.     -   f) In one example, the filtered/mapped reconstructed samples         neighboring the “corresponding luma coding block” may be used to         derive the chroma residual scaling factor and/or the model         computation of CCLM.         -   i. In one example, the filtering/mapping process may include             reference smoothing filtering for intra blocks,             post-filtering such as bilateral filter, Hadamard transform             based filter, forward mapping of reshaper domain and so on. -   12. It is proposed that a fixed value may be used to derive the     chroma residual scaling factor for numbers of chroma blocks (such as     CUs or TUs) in the current slice/tile group.     -   a) In one example, the chroma residual scaling factor for N         chroma blocks may be derived by a fix value, wherein N being 1 .         . . total number of chroma blocks in the current slice/tile         group.     -   b) In one example, a fixed value may be used to find the index         of the piecewise linear model to which the value belongs to, and         the chroma residual scaling factor may be then calculated from         the derived piecewise index. In one example, the fixed value may         depend on the internal bit depth for luma samples.     -   c) In one example, a fixed value may be directly used to         represent the chroma residual scaling factor.     -   d) In one example, the fixed value and/or a fixed chroma         residual scaling factor may depend on picture/slice/tile group         type (e.g., I or P or B slices), and/or coding mode (e.g., intra         mode or inter mode) of a block.     -   e) In one example, the fixed value may be different for         different pictures/slices/tile groups/tiles.     -   f) In one example, a fixed chroma residual scaling factor may be         directly used to scale the chroma residues in LMCS.         -   i. In one example, the fixed chroma residual scaling factor             may be signaled in a video unit such as decoding parameter             set (DPS)/SPS/picture parameter set (PPS)/video parameter             set (VPS)/adaption parameter set (APS)/slice header/tile             group header.         -   ii. In one example, the fixed chroma residual scaling factor             for a picture/slice/tile group/tile may depend on the             mapping function (e.g. the piecewise linear functions) for             luma samples reshaping in the picture/slice/tile group/tile.             -   1) In one example, the fixed chroma residual scaling                 factor may depend on the array InvScaleCoeff[ ] and/or                 the array ScaleCoeff[ ] as defined in JVET-M1001-v7.                 -   a) For example, the array InvScaleCoeff[ ] and/or                     the array ScaleCoeff[ ] to be depended by the fixed                     chroma residual scaling factor for a                     picture/slice/tile group/tile may be used to reshape                     the luma samples in the picture/slice/tile                     group/tile.             -   2) In one example, the fixed chroma residual scaling                 factor may be derived by the                 minimum/maximum/median/average value among all scaling                 coefficients of the piecewise linear functions used to                 reshape the luma samples.                 -   a) For example, the fixed chroma residual scaling                     factor may be derived by the                     minimum/maximum/median/average value of                     InvScaleCoeff[i], wherein i is from a minimum index                     (such as lmcs_min_bin_idx in JVET-M1001-v7) to a                     maximum index (such as LmcsMaxBinIdx in                     JVET-M1001-v7).                 -    i. For example, the fixed chroma residual scaling                     factor may be equal to the                     minimum/maximum/median/average value of                     InvScaleCoeff[i], wherein i is from a minimum index                     (such as lmcs_min_bin_idx defined in JVET-M1001-v7)                     to a maximum index (such as LmcsMaxBinIdx defined in                     JVET-M1001-v7).                 -   ii. Alternatively, the fixed chroma residual scaling                     factor may be derived by the                     minimum/maximum/median/average value of                     InvScaleCoeff[i], wherein i is from a minimum index                     (such as 0) to a maximum index (such as 15), and                     lmcsCW[i] as defined in JVET-M1001-v7 is not equal                     to 0.     -   g) In one example, a fixed value may be used to represent the         piecewise function index to derive a chroma scaling factor.         -   i. In one example, the fixed value may be signaled in a             video unit such as DPS/SPS/PPS/VPS/APS/slice header/tile             group header.         -   ii. In one example, the fixed value may represent a luma             sample intensity, which may be used to identify the             piecewise function index, and the identified piecewise             function index is further used to find the chroma scaling             factor. In one example, the fixed value may equal to 0, or             (1<<lumaBitDepth)−1, or 1<<(lumaBitDepth−1), where             lumaBitDepth denotes the bit depth of luma samples.             -   1) In one example, a piecewise function is identified if                 the fixed value is in the input-range of the piecewise                 function.             -   2) In one example, a piecewise function is identified if                 the fixed value is in the output-range of the piecewise                 function.         -   iii. In one example, the fixed value may represent a fixed             piecewise function index, which may be used to find the             chroma scaling factor.             -   1) In one example, the fixed value may be equal to the                 minimum bin index (e.g., lmcs_min_bin_idx) used in the                 LMCS construction process.             -   2) In one example, the fixed value may be equal to the                 maximum bin index (e.g., LmcsMaxBinIdx) used in the LMCS                 construction process.             -   3) In one example, the fixed value may be equal to the                 average of the maximum bin index and the minimum bin                 index used in the LMCS construction process.                 Restriction on Whether Chroma Residual Scaling and/or                 CCLM is Applied or not -   13. It is proposed that whether the chroma residual scaling or CCLM     is applied or not may depend on the partition of the corresponding     and/or the collocated luma block.     -   a) In one example, whether to enable or disable tools with         cross-component information may depend on the number of         CU/PU/TUs within the collocated luma (e.g., Y or G component)         block.         -   i. In one example, if the number of CU/PU/TUs within the             collocated luma (e.g., Y or G component) block exceeds a             number threshold, such tools may be disabled.         -   ii. Alternatively, whether to enable or disable tools with             cross-component information may depend on the partition tree             depth.             -   1) In one example, if the maximum (or minimum or average                 or other variation) quadtree depth of CUs within the                 collocated luma block exceeds a threshold, such tools                 may be disabled.             -   2) In one example, if the maximum (or minimum or average                 or other variation) BT and/or TT depth of CUs within the                 collocated luma block exceeds a threshold, such tools                 may be disabled.         -   iii. Alternatively, furthermore, whether to enable or             disable tools with cross-component information may depend on             the block dimension of the chroma block.         -   iv. Alternatively, furthermore, whether to enable or disable             tools with cross-component information may depend on whether             the collocated luma cross multiple VPDUs/pre-defined region             sizes.         -   v. The thresholds in the above discussion may be fixed             numbers, or may be signaled, or may be dependent on standard             profiles/levels/tiers.     -   b) In one example, if the collocated luma block of current         chroma block is divided by multiple partitions (e.g., in FIG. 7         ), then the chroma residual scaling and/or CCLM may be         prohibited.         -   i. Alternatively, if the collocated luma block of current             chroma block is not split (e.g., within one CU/TU/PU), then             the chroma residual scaling and/or CCLM may be applied.     -   c) In one example, if the collocated luma block of current         chroma block contains more than M CUs/PUs/TUs, then the chroma         residual scaling and/or CCLM may be prohibited.         -   i. In one example, M may be an integer great than 1.         -   ii. In one example, M may be dependent on whether it is CCLM             or chroma residual scaling process.         -   iii. M may be fixed numbers, or may be signaled, or may be             dependent on standard profiles/levels/tiers.     -   d) The above-mentioned CUs within the collocated luma block may         be interpreted to be all CUs within the collocated luma block.         Alternatively, CUs within the collocated luma block may be         interpreted to be partial CUs within the collocated luma block,         such as CUs along the boundary of the collocated luma block.     -   e) The above-mentioned CUs within the collocated luma block may         be interpreted to be sub-CUs or sub-blocks.         -   i. For example, sub-CUs or sub-blocks may be used in ATMVP;         -   ii. For example, sub-CUs or sub-blocks may be used in affine             prediction;         -   iii. For example, sub-CUs or sub-blocks may be used in Intra             Sub-Partitions (ISP) mode.     -   f) In one example, if the CU/PU/TU covering the top-left luma         sample of the collocated luma block is larger than a pre-defined         luma block size, then the chroma residual scaling and/or CCLM         may be prohibited.         -   i. An example is depicted in FIG. 8 , the collocated luma             block is 32×32 but it is within a corresponding luma block             with size equal to 64×64, then if the pre-defined luma block             size is 32×64, the chroma residual scaling and/or CCLM is             prohibited in this case         -   ii. Alternatively, if the collocated of current chroma block             is not split, and the corresponding luma block covering the             top-left luma sample of the collocated luma blocks is             completely included within a pre-defined bounding box, then             the chroma residual scaling and/or CCLM for current chroma             block may be applied. The bounding box may be defined as a             rectangle with width W and height H, denoted by W×H, as             shown in FIG. 9 , where the corresponding luma block is with             width 32 and height 64, and the bounding box is with width             40 and height 70.             -   1) In one example, the size W×H of the bounding box may                 be defined according to the CTU width and/or height, or                 according to the CU width and/or height, or according to                 arbitrary values.     -   g) In one example, if the collocated luma block of current         chroma block is divided by multiple partitions, then only the         prediction samples (or reconstructed samples) inside the         pre-defined partition of the collocated luma block are used to         derive the chroma residual scaling factor in LMCS mode.         -   i. In one example, the average of all the prediction samples             (or reconstructed samples) in the first partition of the             collocated luma block are used to derive the chroma residual             scaling factor in the LMCS mode.         -   ii. Alternatively, the top-left prediction sample (or             reconstructed sample) in the first partition of the             collocated luma block is used to derive the chroma residual             scaling factor in the LMCS mode.         -   iii. Alternatively, the center prediction sample (or             reconstructed sample) in the first partition of the             collocated luma block is used to derive the chroma residual             scaling factor in the LMCS mode.     -   h) It is proposed that whether and how to apply the         cross-component tools such as CCLM and LMCS may depend on the         coding mode(s) of one or multiple luma CUs, which cover at least         one sample of the collocated luma block.         -   i. For example, the cross-component tools are disabled if             one or multiple luma CUs which cover at least one sample of             the collocated luma block are coded with affine mode;         -   ii. For example, the cross-component tools are disabled if             one or multiple luma CUs which cover at least one sample of             the collocated luma block are coded with bi-prediction;         -   iii. For example, the cross-component tools are disabled if             one or multiple luma CUs which cover at least one sample of             the collocated luma block are coded with BDOF;         -   iv. For example, the cross-component tools are disabled if             one or multiple luma CUs which cover at least one sample of             the collocated luma block are coded with DMVR;         -   v. For example, the cross-component tools are disabled if             one or multiple luma CUs which cover at least one sample of             the collocated luma block are coded with matrix affine             prediction mode as proposed in JVET-N0217;         -   vi. For example, the cross-component tools are disabled if             one or multiple luma CUs which cover at least one sample of             the collocated luma block are coded with inter mode;         -   vii. For example, the cross-component tools are disabled if             one or multiple luma CUs which cover at least one sample of             the collocated luma block are coded with ISP mode;         -   viii. In one example, “one or multiple luma CUs which cover             at least one sample of the collocated luma block” may refer             the corresponding luma block.     -   i) When CCLM/LMCS is prohibited, signaling of the indication of         usage of CCLM/LMCS may be skipped.     -   j) In this disclosure, CCLM may refer to any of the variant         modes of CCLM, including linear model (LM) mode, LM-T mode, and         LM-L mode. -   14. It is proposed that whether and how to apply the cross-component     tools such as CCLM and LMCS may be performed on part of a chroma     block.     -   a) In one example, whether and how to apply the cross-component         tools such as CCLM and LMCS at chroma sub-block level.         -   i. In one example, a chroma sub-block is defined as a 2×2 or             4×4 block in a chroma CU.         -   ii. In one example, for a chroma sub-block, when the             corresponding luma coding block of the current chroma CU             covers all samples of the corresponding block of the             sub-block, CCLM may be applied.         -   iii. In one example, for a chroma sub-block, when not all             samples of the corresponding block are covered by the             corresponding luma coding block of the current chroma CU,             CCLM is not applied.         -   iv. In one example, the parameters of CCLM or LMCS are             derived for each chroma sub-block as treating the sub-block             as a chroma CU.         -   v. In one example, when CCLM or LMCS are applied for a             chroma sub-block, the collocated block's samples may be             used. -   15. It is proposed that whether and how to apply the cross-component     tools such as CCLM may depend on the block dimensions and/or     partition information and/or partition structure type (e.g., dual     tree or single tree).     -   a) In one example, CCLM may be disabled for chroma dual-tree         partition.     -   b) In one example, CCLM may be disabled conditionally depending         on block dimensions.         -   i. In one example, suppose the current block dimensions are             WB×HB, the following CCLM may be disabled depending on the             relationship between WB, HB and two integers T1 and T2.             -   1) In one example, CCLM may be disabled if WB>=T1 and                 HB>=T2. E.g. T1=T2=8.             -   2) In one example, CCLM may be disabled if WB*HB>=T1.                 E.g. T1=64.             -   3) In one example, CCLM may be disabled if Min(WB,                 HB)>=T1. E.g. T1=8.             -   4) In one example, CCLM may be disabled if Max(WB,                 HB)>=T1. E.g. T1=8.             -   5) In one example, CCLM may be disabled if WB<=T1 and                 HB<=T2. E.g. T1=T2=16.             -   6) In one example, CCLM may be disabled if WB*HB<=T1.                 E.g. T1=4096.             -   7) In one example, CCLM may be disabled if Min(WB,                 HB)<=T1. E.g. T1=64.             -   8) In one example, CCLM may be disabled if Max(WB,                 HB)<=T1. E.g. T1=64.     -   c) In one example, CCLM may be restricted by block dimensions         and/or partition information.         -   i. In one example, for a chroma block with dimensions WB×HB,             if one or multiple of below conditions is satisfied, then             CCLM is disabled. For example, WB=HB=32. In the following             bullets, the collocated luma block dimensions may be             (sh×WB)×(sv'HB), where sh and sv are scaling factors. For             example, sh=sv=2 when the color format is 4:2:0.             -   1) Current chroma block is at a leaf node with a                 horizontal split, and the collocated luma block is at a                 leaf node with a vertical split.                 -   a) In one example, current chroma block is split                     with SPLIT_TT_HOR mode, and the collocated luma                     block is split with SPLIT_TT_VER mode.                 -   b) In one example, current chroma block is split                     with SPLIT_TT_HOR mode, and the collocated luma                     block is split with SPLIT_BT_VER mode.                 -   c) In one example, current chroma block is split                     with SPLIT_BT_HOR mode, and the collocated luma                     block is split with SPLIT_TT_VER mode.                 -   d) In one example, current chroma block is split                     with SPLIT_BT_HOR mode, and the collocated luma                     block is split with SPLIT_BT_VER mode.             -   2) Current chroma block is at a leaf node with a                 vertical split, and the collocated luma block is at a                 leaf node with a horizontal split.                 -   a) In one example, current chroma block is split                     with SPLIT_TT_VER mode, and the collocated luma                     block is split with SPLIT_TT_HOR mode.                 -   b) In one example, current chroma block is split                     with SPLIT_TT_VER mode, and the collocated luma                     block is split with SPLIT_BT_HOR mode.                 -   c) In one example, current chroma block is split                     with SPLIT_BT_VER mode, and the collocated luma                     block is split with SPLIT_TT_HOR mode.                 -   d) In one example, current chroma block is split                     with SPLIT_BT_VER mode, and the collocated luma                     block is split with SPLIT_BT_HOR mode.     -   d) In one example, if CCLM is disabled for a block, the syntax         elements related to CCLM may not be signaled in the block.         -   i. Alternatively, if CCLM is disabled for a block, the             syntax elements related to CCLM may be signaled but the CCLM             modes should not be used in the block in a conformance             bit-stream.         -   ii. Alternatively, if CCLM is disabled for a block, the             syntax elements related to CCLM may be signaled. If CCLM             modes are signaled, a default mode such as planar or DC may             be applied.

Applicability of Chroma Residual Scaling in LMCS Mode

-   16. It is proposed that whether luma dependent chroma residual     scaling can be applied may be signaled at other syntax level in     addition to the tile group header as specified in JVET-M1001.     -   a) For example, a chroma_residual_scale_flag may be signaled at         sequence level (e.g. in SPS), at picture level (e.g. in PPS or         picture header), at slice level (e.g. in slice header), at tile         level, at CTU row level, at CTU level, at CU level.         chroma_residual_scale_flag equal to 1 specifies that chroma         residual scaling is enabled for the CUs below the signaled         syntax level. chroma_residual_scale_flag equal to 0 specifies         that chroma residual scaling is not enabled for below the         signaled syntax level. When chroma_residual_scale_flag is not         present, it is inferred to be equal to 0.     -   b) In one example, if chroma residual scaling is constrained at         a partition node level. Then chroma_residual_scale_flag may not         be signaled and inferred to be 0 for CUs covered by the         partition node. In one example, a partition node may be a CTU         (CTU is treated as the root node of quaternary tree partition).     -   c) In one example, if chroma residual scaling is constrained for         chroma block size equal or smaller than 32×32, then         chroma_residual_scale flag may not be signaled and inferred to         be 0 for chroma block size equal or smaller than 32×32.

Applicability of CCLM Mode

-   17. It is proposed that whether CCLM mode can be applied may be     signaled at other syntax levels in addition to the SPS level as     specified in JVET-M1001.     -   a) For example, it may be signaled at picture level (e.g. in PPS         or picture header), at slice level (e.g. in slice header), at         tile group level (e.g. in tile group header), at tile level, at         CTU row level, at CTU level, at CU level.     -   b) In one example, cclm_flag may not be signaled and inferred to         be 0 if CCLM cannot be applied.         -   i. In one example, if chroma residual scaling is constrained             for chroma block size equal or smaller than 8×8, then             cclm_flag may not be signaled and inferred to be 0 for             chroma block size equal or smaller than 8×8.

Unification of Chroma Residual Scaling Factor Derivation for Intra Mode and Inter Mode

-   18. Chroma residual scaling factor may be derived after     encoding/decoding a luma block and may be stored and used for     following coded blocks.     -   a) In one example, certain prediction samples or/and         intermediate prediction samples or/and reconstructed samples         or/and reconstructed samples before loop filtering (e.g., before         processed by deblocking filter or/and SAO filter or/and         bilateral filter or/and Hadamard transform filter or/and ALF         filter) in the luma block may be used for derivation of the         chroma residual scaling factor.         -   i. For example, partial samples in the bottom row or/and             right column of the luma block may be used for derivation of             the chroma residual scaling factor.     -   b) In single tree case, when encoding a block coded in intra         mode or/and IBC mode or/and inter mode, derived chroma residual         scaling factor of neighboring blocks may be used for deriving         scaling factor of the current block.         -   i. In one example, certain neighboring blocks may be checked             in order, and the first available chroma residual scaling             factor may be used for the current block.         -   ii. In one example, certain neighboring blocks may be             checked in order, and a scaling factor may be derived based             on the first K available neighboring chroma residual scaling             factors.             -   iii. In one example, for a block coded in inter mode                 or/and CIIP mode, if a neighboring block is coded in                 intra mode or/and IBC mode or/and CIIP mode, chroma                 residual scaling factor of the neighboring block may be                 considered as “unavailable”.             -   iv. In one example, neighboring blocks may be checked in                 order of left (or above left)->above (or above right).                 -   1) Alternatively, neighboring blocks may be checked                     in order of above (or above right)->left (or above                     left).     -   c) In separate tree case, when encoding a chroma block, the         corresponding luma block may be first identified. Then, derived         chroma residual scaling factor of its (e.g., the corresponding         luma block) neighboring blocks may be used for deriving scaling         factor of the current block.         -   i. In one example, certain neighboring blocks may be checked             in order, and the first available chroma residual scaling             factor may be used for the current block.         -   ii. In one example, certain neighboring blocks may be             checked in order, and a scaling factor may be derived based             on the first K available neighboring chroma residual scaling             factors.     -   d) Neighboring blocks may be checked in a predefined order.         -   i. In one example, neighboring blocks may be checked in             order of left (or above left)->above (or above right).         -   ii. In one example, neighboring blocks may be checked in             order of above (or above right)->left (or above left).         -   iii. In one example, neighboring blocks may be checked in             order of below left->left->above right->above->above left.         -   iv. In one example, neighboring blocks may be checked in             order of left->above->above right->below left->above left.     -   e) In one example, whether to apply chroma residual scaling may         depend on the “availability” of neighbouring block.         -   i. In one example, when there is no “available” neighbouring             block, chroma residual scaling may be disallowed.         -   ii. In one example, when the number of “available”             neighbouring blocks is smaller than K (K>=1), chroma             residual scaling may be disallowed.         -   iii. Alternatively, when there is no “available”             neighbouring block, chroma residual scaling factor may be             derived by a default value.             -   1) In one example, a default value 1<<(BitDepth−1) may                 be used to derive the chroma residual scaling factor.     -   f) In one example, the chroma residual scaling factor of current         chroma block may be stored and used for following coded blocks.     -   g) In one example, the storage of chroma residual scaling         factors may be removed from line buffer.         -   i. In one example, when the current block and a neighboring             (adjacent or non-adjacent) block to be accessed are in             different regions, its chroma residual scaling factor may be             considered as “unavailable” and may not be used for the             derivation of chroma residual scaling factor of the current             block.             -   1) A region may be a slice, a tile, a tile group, a CTU                 row, or a CTU.             -   2) Alternatively, its chroma residual scaling factor may                 be considered as a default value in such a case.             -   3) Alternatively, chroma residual scaling cannot be                 applied in such a case.     -   h) In one example, the chroma residual scaling factor of current         chroma block may be updated on-the-fly and may be saved in a         history table for scaling factor derivation of the following         blocks.         -   i. The history table may be updated in a FIFO (first-in             first-out) way.         -   ii. After decoding/encoding a chroma block, a chroma             residual scaling factor may be derived (e.g., according to             the luma values) and may be stored in the FIFO history             table.         -   iii. In one example, the FIFO history table may contain at             most 1 entry. In this case, derived chroma residual scaling             factor of the latest decoded block is used for the current             block.         -   iv. In one example, the history table is refreshed before             encoding/decoding a picture, and/or a slice, and/or a tile             group, and/or a tile, and/or a CTU row and/or a CTU.             -   1) In one example, a default chroma residual scaling                 factor may be put into the history table when the                 history table is refreshed.             -   2) In one example, the history table is set to empty                 when the FIFO history table is refreshed.

5. Embodiments 5.1 Embodiment #1

The example embodiment discussed below is for an illustrative method in item 11 of Section 4. Newly added parts are shown in underline and bold, and the deleted parts from VVC working draft are shown in bold capitalized font. The modifications are based on the latest VVC working draft (JVET-M1007-v7) and the new adoption in JET-N220-v3.

8.7.5.4 Picture Reconstruction with Luma Dependent Chroma Residual Scaling Process for Chroma Samples

Inputs to this process are:

-   -   a location (xCurr, yCurr) of the top-left sample of the current         transform block relative to the top-left sample of the current         picture,     -   a variable nCurrSw specifying the transform block width,     -   a variable nCurrSh specifying the transform block height,     -   an (nCurrSw)×(nCurrSh) array predSamples specifying the chroma         prediction samples of the current block,     -   an (nCurrSw)×(nCurrSh) array resSamples specifying the chroma         residual samples of the current block.         Output of this process is a reconstructed chroma picture sample         array recSamples.         The current luma location (xTbY, yTbY) is set to (xCurr<<1,         vCurr<<1)         The corresponding luma location (xTbCorrY, yTbCorrY) is derived         as follows:     -   If tile group type is equal to 2 (I tile group) and qtbtt dual         tree intra flag is equal to 1, the corresponding luma location         (xTbCorrY, yTbCorrY) is set to the top-left sample location of         the corresponding CU that covers the top-left sample of the         current luma location (xTbY, yTbY).     -   Otherwise, the corresponding luma location (xTbCorrY, yTbCorrY)         is set to the current luma location (xTbY, yTbY)         The variable availL is derived as follows:     -   The availability of left neighbouring sample derivation process         for a block as specified in clause 6.4.X [Ed. (BB): Neighbouring         blocks availability checking process tbd] is invoked with the         corresponding luma location (xTbCorrY, yTbCorrY) and the         neighbouring luma location (xTbCorrY−1, yTbCorrY) as inputs, and         the output is assigned to availL.         When availL is equal to TRUE, the neighbouring top-left luma         sample pY[x][y] with x=−1, y=0, is set equal to the         reconstructed luma samples prior to the deblocking filter         process at the location (xTbCorrY+x, yTbCorrY+y). The mapped         value of the neighbouring top-left luma sample pYMap[x][y] with         x=−1, y=−1, is derived as follows:

idxY=pY[x][y]>> Log 2(OrgCW)

pYMap[x][y]=Clip1_(Y)(LmcsPivot[idxY]+(ScaleCoeff[idxY]*(pY[x][y]−InputPivot[idxY])+(1<<10))>>11)

The reconstructed chroma picture sample recSamples is derived as follows for i=0 . . . nCurrSw−1, j=0 . . . nCurrSh−1:

-   -   IF TILE_GROUP_CHROMA_RESIDUAL_SCALE_FLAG IS EQUAL TO 0 OR         NCURRSW*NCURRSH IS LESS THAN OR EQUAL TO 4, THE FOLLOWING         APPLIES:

RECSAMPLES[XCURR+I][YCURR+J]=CLIP1_(C)(PREDSAMPLES[I][J]+RECSAMPLES[I][J])  (8-1063)

-   -   If one of the following conditions is true,         recSamples[xCurr+i][yCurr+j] is set equal to     -   Clip1_(C)(predSamples[i][j]+recSamples[i][j])         -   tile_group_chroma_residual_scale_flag is equal to 0         -   nCurrSw*nCurrSh is less than or equal to 4         -   availL is equal to false         -   CuPredMode[xCurr][yCurr] is equal to MODE INTER and ciip             flag[xCurr][yCurr] is equal to 0, and             CuPredMode[xTbCorrY−1][yTbCorrY] is equal to MODE INTRA         -   CuPredMode[xCurr][yCurr] is equal to MODE INTER and ciip             flag[xCurr][yCurr] is equal to 0, and             CuPredMode[xTbCorrY−1][yTbCorrY] is equal to MODE IBC         -   CuPredMode[xCurr][yCurr] is equal to MODE INTER and ciip             flag[xCurr][yCurr] is equal to 0, and             CuPredMode[xTbCorrY−1][yTbCorrY] is equal to MODE INTER and             ciip flag[xCurr][yCurr] is equal to 1     -   Otherwise (tile_group_chroma_residual_scale_flag is equal to 1         and nCurrSw*nCurrSh is greater than 4), the following applies:         -   For the derivation of the variable varScale the following             ordered steps apply:             -   1. The variable invAvgLuma is derived as follows:

INVAVGLUMA=CLIP1_(Y)((Σ_(k=0) ^(2+nCurrSw−1)Σ_(l=0) ^(2+nCurrSh−1)PredMapSamples[k][l]+NCURRSW*NCURRSH*2)/(NCURRSW*NCURRSH*4))

invAvgLuma=Clip1_(Y)(pYMap[−1][1])  (8-1064)

The variable idxYInv is derived by invoking the identification of piece-wise function index as specified in clause 8.7.5.3.2 with invAvgLuma as the input and idxYInv as the output.

-   -   -   -   2. The variable varScale is derived as follows:

varScale=ChromaScaleCoeff[idxYInv]  (8-1065)

-   -   -   -   The recSamples is derived as follows:                 -   If tu_cbf_cIdx[xCurr][yCurr] equal to 1, the                     following applies:

resSamples[i][j]=Clip3((1<<BitDepth_(C)),1<<BitDepth_(C)−1,resSamples[i][j])

recSamples[xCurr+i][yCurr+j]=ClipCidx1(predSamples[i][j]+Sign(resSamples[i][j])*((Abs(resSamples[i][j])*varScale+(1<<10))>>11))  (8-1067)

-   -   -   -   -   Otherwise (tu_cbf_cIdx[xCurr][yCurr] equal to 0),                     the following applies:

recSamples[xCurr+i][yCurr+j]=ClipCidx1(predSamples[i][j])  (8-1067)

5.2 Embodiments #2

The embodiment below is for the method in item 11 of the invention description. Newly added parts are shown in underscore and bold, and the deleted parts from VVC working draft are shown in bold capitalized font. The modifications are based on the latest VVC working draft (JVET-M1007-v7) and the new adoption in JVET-N220-v3. The differences between Embodiment #2 and #1 are listed as follows:

-   -   Multiple neighbor luma samples are checked to derive the chroma         residual scaling factor.     -   When the neighbor luma samples are not available, or when         neighbor luma is coded in INTRA/CIIP/IBC mode while current is         coded in INTER mode, #2 uses a default value for chroma residual         scaling factor derivation.

8.7.5.4 Picture Reconstruction with Luma Dependent Chroma Residual Scaling Process for Chroma Samples

Inputs to this process are:

-   -   a location (xCurr, yCurr) of the top-left sample of the current         transform block relative to the top-left sample of the current         picture,     -   a variable nCurrSw specifying the transform block width,     -   a variable nCurrSh specifying the transform block height,     -   an (nCurrSw)×(nCurrSh) array predSamples specifying the chroma         prediction samples of the current block,     -   an (nCurrSw)×(nCurrSh) array resSamples specifying the chroma         residual samples of the current block.         Output of this process is a reconstructed chroma picture sample         array recSamples.         The current luma location (xTbY, yTbY) is set to (xCurr<<1,         yCurr<<1)         The corresponding luma location (xTbCorrY, yTbCorrY) is derived         as follows:     -   If tile group type is equal to 2 (I tile group) and qtbtt dual         tree intra flag is equal to 1, the corresponding luma location         (xTbCorrY, yTbCorrY) is set to the top-left sample location of         the corresponding CU that covers the top-left sample of the         current luma location (xTbY, yTbY).     -   Otherwise, the corresponding luma location (xTbCorrY, yTbCorrY)         is set to the current luma location (xTbY, yTbY)         The variable availTL is derived as follows:     -   The availability of top-left neighbouring sample derivation         process for a block as specified in clause 6.4.X [Ed. (BB):         Neighbouring blocks availability checking process tbd] is         invoked with the corresponding luma location (xTbCorrY,         yTbCorrY) and the neighbouring luma location (xTbCorrY−1,         yTbCorrY−1) as inputs, and the output is assigned to availTL.         The variable availT is derived as follows:     -   The availability of top neighbouring sample derivation process         for a block as specified in clause 6.4.X [Ed. (BB): Neighbouring         blocks availability checking process tbd] is invoked with the         corresponding luma location (xTbCorrY, yTbCorrY) and the         neighbouring luma location (xTbCorrY, yTbCorrY−1) as inputs, and         the output is assigned to availT.         The numAvailNeigh is set equal to availTL+availT.         If numAvailNeigh is larger or equal to 1, the neighbouring luma         samples pY[x][y] with x=−1, 0, y=−1, are set equal to the         average value of the reconstructed luma samples prior to the         deblocking filter process at the location (xTbCorrY+x,         yTbCorrY+y). The average value of neighbouring luma samples         pY_(Avg) is derived as follows:

pY _(Avg)=((availTL?pY[−1][−1]:0)+(availT?pY[0][−1]:0))>>(numAvailNeigh>1?1:0)

Otherwise, if one of the following conditions is true, pY_(Avg) is set equal to (1<<(BitDepth_(Y)−1)).

-   -   availTL is equal to false     -   CuPredMode[xCurr][yCurr] is equal to MODE INTER and ciip         flag[xCurr][yCurr] is equal to 0, and         CuPredMode[xTbCorrY][yTbCorrY−1] is equal to MODE INTRA     -   CuPredMode[xCurr][yCurr] is equal to MODE INTER and ciip         flag[xCurr][yCurr] is equal to 0, and         CuPredMode[xTbCorrY−1][yTbCorrY−1] is equal to MODE IBC     -   CuPredMode[xCurr][yCurr] is equal to MODE INTER and ciip         flag[xCurr][yCurr] is equal to 0, and         CuPredMode[xTbCorrY−1][yTbCorrY−1] is equal to MODE INTER and         ciip flag[xCurr][yCurr] is equal to 1         The mapped value of the neighbouring top-left luma sample         pYMap_(Avg), is derived as follows:

idxY=pY_(Avg)>> Log 2(OrgCW)

pYMap_(Avg)=Clip1_(Y)(LmcsPivot[idxY]+(ScaleCoeff[idxY]*(pY _(Avg)−InputPivot[idxY])+(1<<10))>>11)

The reconstructed chroma picture sample recSamples is derived as follows for i=0 . . . nCurrSw−1, j=0 . . . nCurrSh−1:

-   -   If tile_group_chroma_residual_scale_flag is equal to 0 or         nCurrSw*nCurrSh is less than or equal to 4, the following         applies:

recSamples[xCurr+i][yCurr+j]=Clip1_(C)(predSamples[i][j]+recSamples[i][j])   (8-1063)

-   -   Otherwise (tile_group_chroma_residual_scale_flag is equal to 1         and nCurrSw*nCurrSh is greater than 4), the following applies:         -   For the derivation of the variable varScale the following             ordered steps apply:             -   The variable invAvgLuma is derived as follows:

INVAVGLUMA=CLIP1_(Y)((Σ_(k=0) ^(2+nCurrSw−1)Σ_(l=0) ^(2+nCurrSh−1)PredMapSamples[k][l]+NCURRSW*NCURRSH*2)/(NCURRSW*NCURRSH*4))

invAvgLuma=Clip1^(Y)(pYMap_(Avg))  (8-1064)

The variable idxYInv is derived by invoking the identification of piece-wise function index as specified in clause 8.7.5.3.2 with invAvgLuma as the input and idxYInv as the output.

-   -   The variable varScale is derived as follows:

varScale=ChromaScaleCoeff[idxYInv]  (8-1065)

-   -   The recSamples is derived as follows:         -   If tu_cbf_cIdx[xCurr][yCurr] equal to 1, the following             applies:

resSamples[i][j]=Clip3(−(1<<BitDepth_(C)),1<<BitDepth_(C)−1,resSamples[i][j])

recSamples[xCurr+i][yCurr+j]=ClipCidx1(predSamples[i][j]+Sign(resSamples[i][j])*((Abs(resSamples[i][j])*varScale+(1<<10))>>11))  (8-1066)

-   -   Otherwise (tu_cbf_cIdx[xCurr][yCurr] equal to 0), the following         applies:

recSamples[xCurr+i][yCurr+j]=ClipCidx1(predSamples[i][j])  (8-1067)

5.3 Embodiment #3

The embodiment below is for the method in item 12 of the invention description. Newly added parts are highlighted in bold italics, and the deleted parts from VVC working draft are highlighted in capitalized bold fonts. The modifications are based on the latest VVC working draft (JVET-M1007-v7) and the new adoption in JVET-N220-v3.

8.7.5.4 Picture Reconstruction with Luma Dependent Chroma Residual Scaling Process for Chroma Samples

Inputs to this process are:

-   -   a location (xCurr, yCurr) of the top-left sample of the current         transform block relative to the top-left sample of the current         picture,     -   a variable nCurrSw specifying the transform block width,     -   a variable nCurrSh specifying the transform block height,     -   an (nCurrSw)×(nCurrSh) array predSamples specifying the chroma         prediction samples of the current block,     -   an (nCurrSw)×(nCurrSh) array resSamples specifying the chroma         residual samples of the current block.         Output of this process is a reconstructed chroma picture sample         array recSamples.         The reconstructed chroma picture sample recSamples is derived as         follows for i=0 . . . nCurrSw−1, j=0 . . . nCurrSh−1:     -   If slice_chroma_residual_scale_flag is equal to 0 or         nCurrSw*nCurrSh is less than or equal to 4, the following         applies:

recSamples[xCurr+i][yCurr+j]=Clip1_(C)(predSamples[i][j]+resSamples[i][j])  (8-1063)

-   -   Otherwise (slice_chroma_residual_scale_flag is equal to 1 and         nCurrSw*nCurrSh is greater than 4), the following applies:         -   For the derivation of the variable varScale the following             ordered steps apply:         -   1. THE VARIABLE INVAVGLUMA IS DERIVED AS FOLLOWS:

INVAVGLUMA=CLIP1_(Y)((Σ_(k=0) ^(2*nCurrSw−1)Σ_(l=0) ^(2*nCurrSh−1)PredMapSamples[k][l]+NCURRSW*NCURRSH*2)/(NCURRSW*NCURRSH*4)  (8-993)

-   -   -   2. THE VARIABLE IDXYINV IS DERIVED BY INVOKING THE             IDENTIFICATION OF PIECE-WISE FUNCTION INDEX AS SPECIFIED IN             CLAUSE 8.7.5.3.2 WITH INVAVGLUMA AS THE INPUT AND IDXYINV AS             THE OUTPUT.         -   3. The variable varScale is derived as follows:

varScale=ChromaScaleCoeff[[[idxYInv]]lmcs_min_bin_idx]  (8-1064)

-   -   The recSamples is derived as follows:         -   If tu_cbf_cIdx[xCurr][yCurr] equal to 1, the following             applies:

resSamples[i][j]=Clip3(−(1<<BitDepth_(C)),1<<BitDepth_(C)−1,resSamples[i][j])

recSamples[xCurr+i][yCurr+j]=ClipCidx1(predSamples[i][j]+Sign(resSamples[i][j])*((Abs(resSamples[i][j])*varScale+(1<<10))>>11))  (8-1066)

-   -   -   Otherwise (tu_cbf_cIdx[xCurr][yCurr] equal to 0), the             following applies:

recSamples[xCurr+i][yCurr+j]=ClipCidx1(predSamples[i][j])  (8-1067)

5.4 Embodiment #4

The embodiment below is for the method in item 12 of the invention description. Newly added parts are highlighted in bold italics, and the deleted parts from VVC working draft are highlighted in capitalized bold font. The modifications are based on the latest VVC working draft (JVET-M1007-v7) and the new adoption in JVET-N220-v3.

8.7.5.4 Picture Reconstruction with Luma Dependent Chroma Residual Scaling Process for Chroma Samples

Inputs to this process are:

-   -   a location (xCurr, yCurr) of the top-left sample of the current         transform block relative to the top-left sample of the current         picture,     -   a variable nCurrSw specifying the transform block width,     -   a variable nCurrSh specifying the transform block height,     -   an (nCurrSw)×(nCurrSh) array predSamples specifying the chroma         prediction samples of the current block,     -   an (nCurrSw)×(nCurrSh) array resSamples specifying the chroma         residual samples of the current block.         Output of this process is a reconstructed chroma picture sample         array recSamples.         The reconstructed chroma picture sample recSamples is derived as         follows for i=0 . . . nCurrSw−1, j=0 . . . nCurrSh−1:     -   If slice_chroma_residual_scale_flag is equal to 0 or         nCurrSw*nCurrSh is less than or equal to 4, the following         applies:

recSamples[xCurr+i][yCurr+j]=Clip1_(C)(predSamples[i][j]+resSamples[i][j])   (8-1063)

-   -   Otherwise (slice_chroma_residual_scale_flag is equal to 1 and         nCurrSw*nCurrSh is greater than 4), the following applies:         -   For the derivation of the variable varScale the following             ordered steps apply:         -   1. The variable invAvgLuma is derived as follows:

INVAVGLUMA=CLIP1_(Y)((Σ_(k=0) ^(2*nCurrSw−1)Σ_(l=0) ^(2*nCurrSh−1)PredMapSamples[k][l]+NCURRSW*NCURRSH*2)/(NCURRSW*NCURRSH*4))  (8-993)

invAvgLuma=0  (8-1064)

-   -   -   2. The variable idxYInv is derived by invoking the             identification of piece-wise function index as specified in             clause 8.7.5.3.2 with invAvgLuma as the input and idxYInv as             the output.         -   3. The variable varScale is derived as follows:

varScale=ChromaScaleCoeff[idxYInv]  (8-994)

-   -   The recSamples is derived as follows:         -   If tu_cbf_cIdx[xCurr][yCurr] equal to 1, the following             applies:

resSamples[i][j]=Clip3(−(1<<BitDepth_(C)),1<<BitDepth_(C)−1,resSamples[i][j])

recSamples[xCurr+i][yCurr+j]=ClipCidx1(predSamples[i][j]+Sign(resSamples[i][j])*((Abs(resSamples[i][j])*varScale+(1<<10))>>11))  (8-1066)

-   -   -   Otherwise (tu_cbf_cIdx[xCurr][yCurr] equal to 0), the             following applies:

recSamples[xCurr+i][yCurr+j]=ClipCidx1(predSamples[i][j])  (8-1067)

5.5 Embodiment #5

The embodiment below is for the method in item 12 of the invention description. Newly added parts are highlighted in bold italics, and the deleted parts from VVC working draft are highlighted in double square brackets [[ . . . ]]. The modifications are based on the latest VVC working draft (JVET-M1007-v7) and the new adoption in JVET-N220-v3.

7.4.5.4 Luma Mapping with Chroma Scaling Data Semantics

The variable ChromaScaleCoeff[i], with i=0 . . . 15, is derived as follows:

  if ( lmcsCW[ i ] = = 0 )  ChromaScaleCoeff[ i ] = (1 << 11) else {  ChromaScaleCoeff[ i ] = InvScaleCoeff[ i ] }

 

 

 

 

  

 

8.7.5.4 Picture Reconstruction with Luma Dependent Chroma Residual Scaling Process for Chroma Samples

Inputs to this process are:

-   -   a location (xCurr, yCurr) of the top-left sample of the current         transform block relative to the top-left sample of the current         picture,     -   a variable nCurrSw specifying the transform block width,     -   a variable nCurrSh specifying the transform block height,     -   an (nCurrSw)×(nCurrSh) array predSamples specifying the chroma         prediction samples of the current block,     -   an (nCurrSw)×(nCurrSh) array resSamples specifying the chroma         residual samples of the current block.         Output of this process is a reconstructed chroma picture sample         array recSamples.         The reconstructed chroma picture sample recSamples is derived as         follows for i=0 . . . nCurrSw−1, j=0 . . . nCurrSh−1:     -   If slice_chroma_residual_scale_flag is equal to 0 or         nCurrSw*nCurrSh is less than or equal to 4, the following         applies:

recSamples[xCurr+i][yCurr+j]=Clip1_(C)(predSamples[i][j]+resSamples[i][j])  (8-1063)

-   -   Otherwise (slice_chroma_residual_scale_flag is equal to 1 and         nCurrSw*nCurrSh is greater than 4), the following applies:         -   FOR THE DERIVATION OF THE VARIABLE VARSCALE THE FOLLOWING             ORDERED STEPS APPLY:         -   4. THE VARIABLE INVAVGLUMA IS DERIVED AS FOLLOWS:

INVAVGLUMA=CLIP1_(Y)((Σ_(k=0) ^(2*nCurrSw−1)Σ_(l=0) ^(2*nCurrSh−1)PredMapSamples[k][l]+NCURRSW*NCURRSH*2)/(NCURRSW*NCURRSH*4))  (8-993)

-   -   -   5. THE VARIABLE IDXYINV IS DERIVED BY INVOKING THE             IDENTIFICATION OF PIECE-WISE FUNCTION INDEX AS SPECIFIED IN             CLAUSE 8.7.5.3.2 WITH INVAVGLUMA AS THE INPUT AND IDXYINV AS             THE OUTPUT.         -   6. THE VARIABLE VARSCALE IS DERIVED AS FOLLOWS:

VARSCALE=CHROMASCALECOEFF[IDXYINV]  (8-994)

-   -   -   The recSamples is derived as follows:             -   If tu_cbf_cIdx[xCurr][yCurr] equal to 1, the following                 applies:

resSamples[i][j]=Clip3(−(1<<BitDepth_(C)),1<<BitDepth_(C)−1,resSamples[i][j])

recSamples[xCurr+i][yCurr+j]=ClipCidx1(predSamples[i][j]+Sign(resSamples[i][j])*((Abs(resSamples[i][j])*varScale+(1<<10))>>11))  (8-1066)

-   -   -   -   Otherwise (tu_cbf_cIdx[xCurr][yCurr] equal to 0), the                 following applies:

recSamples[xCurr+i][yCurr+j]=ClipCidx1(predSamples[i][j])  (8-1067)

6. Example Implementations of the Disclosed Embodiments

FIG. 10 is a block diagram of a video processing apparatus 1000. The apparatus 1000 may be used to implement one or more of the methods described herein. The apparatus 1000 may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, and so on. The apparatus 1000 may include one or more processors 1002, one or more memories 1004 and video processing hardware 1006. The processor(s) 1002 may be configured to implement one or more methods (including, but not limited to, methods 800 and 900) described in the present document. The memory (memories) 1004 may be used for storing data and code used for implementing the methods and techniques described herein. The video processing hardware 1006 may be used to implement, in hardware circuitry, some techniques described in the present document.

In some embodiments, the video coding methods may be implemented using an apparatus that is implemented on a hardware platform as described with respect to FIG. 10 .

FIG. 11 shows a flowchart of an example method 1100 for linear model derivations for cross-component prediction in accordance with the disclosed embodiments. The method 1100 includes, at step 1110, performing a conversion between a current video block and a bitstream representation of the current video block, wherein, during the conversion, a second set of color component values of the current video block are derived from a first set of color component values included in one or more reference frames, wherein the first set of color component values are usable in a linear model of a video coding step.

Some embodiments may be described using the following clause-based format.

1. A method for video processing, comprising:

-   -   performing a conversion between a current video block and a         bitstream representation of the current video block, wherein,         during the conversion, a second set of color component values of         the current video block are derived from a first set of color         component values included in one or more reference frames,         wherein the first set of color component values are usable in a         linear model of a video coding step.

2. The method of clause 1, wherein the first set of color component values are interpolated prior to use in the linear model of the video coding step.

3. The method of any one or more of clauses 1-2, wherein a linear combination of the first set of color component values are usable as parameters in the linear model.

4. The method of clause 1, wherein locations of the first set of color component values included in the one or more reference frames are selected based, at least in part, on motion information of the current video block.

5. The method of clause 4, wherein a position of a luma component value in the one or more reference frames is calculated from a position of a corresponding luma component value in the current video block and the motion information of the current video block.

6. The method of clause 5, wherein the position of the corresponding luma component value is a top-left sample, a center sample, or a bottom-right sample in the current video block.

7. The method of clause 6, wherein the motion information of the current video block corresponds to an integer motion vector or a fractional motion vector.

8. The method of clause 7, wherein the fractional motion vector is derived using a fractional luma component value in the one or more reference frames.

9. The method of clause 7, wherein the integer motion vector is derived by rounding towards zero or away from zero.

10. The method of clause 1, wherein locations of the first set of color component values included in the one or more reference frames are pre-defined positions.

11. The method of any one or more of clauses 1-10, wherein a median or an average of the first set of color component values are used to derive the second set of color component values of the current video block.

12. The method of any one or more of clauses 1-11, wherein the one or more reference frames are pre-defined reference frames.

13. The method of clause 12, wherein the pre-defined reference frames include a frame with a reference index of a reference picture list.

14. The method of clause 13, wherein the reference index is zero and the reference picture list is zero.

15. The method of clause 13, wherein the reference index and/or the reference picture list is signaled in the bitstream representation associated with one or more of the following: a sequence, a picture, a tile, a group, a slice, a tile, a coding tree unit row, or a video block.

16. The method of clause 1, wherein the second set of color component values of the current video block are derived from a mathematical mean or a weighted average of the first set of color component values included in the one or more reference frames.

17. The method of clause 1, wherein the second set of color component values of the current video block are selectively derived from the first set of color component values included in the one or more reference frames, based on whether the current video block is a bi-prediction coded block.

18. The method of clause 17, wherein the second set of color component values of the current video block are individually derived for each prediction direction of the first set of color component values.

19. The method of clause 1, wherein the second set of color component values of the current video block are selectively derived from the first set of color component values included in the one or more reference frames, based on whether the current video block is associated with sub-block-based prediction.

20. The method of clause 1, wherein the sub-block-based prediction corresponds to affine prediction or alternative temporal motion vector prediction (ATMVP).

21. The method of any one or more of clauses 19-20, wherein the second set of color component values of the current video block are derived for individual sub-blocks.

22. The method of any one or more of clauses 19-21, wherein the second set of color component values of the current video block are derived for an entirety of the current video block regardless of the sub-block-based prediction.

23. The method of any one or more of clauses 19-22, wherein the first set of color component values included in one or more reference frames are selected based, at least in part on a motion vector of a sub-block of the current video block.

24. The method of any one or more of clauses 1-23, wherein the first set of color component values included in one or more reference frames are intermediate color component values.

25. The method of any one or more of clauses 1-24, wherein the video coding step precedes another video coding step.

26. The method of clause 25, wherein the first set of color component values included in the one or more reference frames are selected based, at least in part on an intermediate motion vector of the current video block or a sub-block of the current video block, and wherein the intermediate motion vector is calculated prior to the other video coding step.

27. The method of any one or more of clauses 24-26, wherein the other video coding step includes one or a combination of the following steps: a Bi-Directional Optical Flow (BDOF) step, a decoder-side motion vector refinement (DMVR) step, a prediction refinement optical flow (PROF) step.

28. The method of any one or more of clauses 1-27, wherein the first set of color component values included in the one or more reference frames correspond to M×N luma component values associated with a corresponding luma block.

29. The method of clause 28, wherein the corresponding luma block is a collocated luma block of the current video block.

30. The method of clause 29, wherein a product of M and N is smaller than a product of a block width and a block height of the collocated luma block of the current video block.

31. The method of any one or more of clauses 27-30, wherein the first set of color component values included in the one or more reference frames correspond to at least a portion of reference samples identified at positions of neighboring luma samples of the collocated luma block.

32. The method of any one or more of clauses 1-31, wherein the first set of color component values are down sampled prior to use in the linear model of the video coding step.

33. The method of clause 1, wherein the second set of color component values of the current video block are selected, based, at least in part on one or more of the following information of the current video block: a quantization parameter, a coding mode, or a picture order count (POC).

34. The method of clause 31, wherein the positions of the neighboring luma samples are such that a top left sample of the collocated luma block is covered.

35. The method of clause 28, wherein the first set of color component values included in the one or more reference frames correspond to at least a portion of reference samples identified at positions external to the corresponding luma block.

36. The method of clause 28, wherein the second set of color component values of the current video block are selectively derived from the first set of color component values included in the one or more reference frames, based on availability of neighboring samples of the corresponding luma block.

37. The method of clause 28, wherein the availability of the neighboring samples of the corresponding luma block is based on one or more of: a use of a coding mode of the current video block, a use of a coding mode of the neighboring samples of the corresponding luma block, a use of a coding mode of the corresponding luma block, a use of a coding mode of one or more neighboring video blocks, a use of a type of a filter associated with the neighboring samples of the corresponding luma block, or a location of the neighboring samples of the corresponding luma block relative to the current video blocks or sub-blocks thereof.

38. The method of clause 28, further comprising:

in response to a lack of the availability of the neighboring samples of the corresponding luma block, substituting, filling, or padding unavailable samples with other samples.

39. The method of clause 28, further comprising:

applying a smoothing filter to samples neighboring the corresponding luma block.

40. A method for video processing, comprising:

-   -   performing a conversion between a current video block and a         bitstream representation of the current video block, wherein,         during the conversion, a second set of color component values of         the current video block are derived from a first set of color         component values included in one or more reference frames,         wherein the first set of color component values are usable in a         linear model of a video coding step; and     -   in response to determining that the first set of color component         values included in the one or more reference frames is a         collocated luma block of the current video block, selectively         enabling or disabling derivation of the second set of color         component values of the current video block, based on one or         more conditions associated with the collocated luma block of the         current video block.

41. The method of clause 40, wherein the one or more conditions associated with the collocated luma block of the current video block include: a partition size of the collocated luma block, a number of coding units of the collocated luma block achieving a threshold number, a top-left luma sample of the collocated luma block achieving a threshold size, a partition tree depth of the collocated luma block, a corresponding luma block covering the top-left luma sample of the collocated luma block, a dimension of the collocated luma block or the current video block, or a corresponding luma block covering the top-left luma sample of the collocated luma block and additionally included within a bounding box of pre-defined size.

42. The method of clause 40, wherein information indicating the selectively enabling or disabling the derivation is included in the bitstream representation.

43. The method of clause 28, wherein the availability of neighboring samples of the corresponding luma block is associated with checking for the neighboring samples according to a pre-defined order.

44. The method of clause 41, wherein the collocated luma block and the current video block are associated with a same coding tree unit or a same row of a coding tree unit.

45. A method for video processing, comprising:

-   -   performing a conversion between a current video block and a         bitstream representation of the current video block, wherein,         during the conversion, a second set of color component values of         the current video block are derived from a first set of color         component values included in one or more reference frames,         wherein the first set of color component values are usable in a         linear model of a video coding step; and     -   in response to determining that one or more properties of the         current video block or neighboring video blocks of the current         video block are satisfied, selectively enabling or disabling         derivation of the second set of color component values of the         current video block.

46. The method of clause 45, wherein the one or more properties of the current video block or neighboring video blocks of the current video block correspond to a neighboring luma block covering a spatial position relative to a spatial position of the current video block.

47. The method of clause 45, wherein the one or more properties of the current video block or neighboring video blocks of the current video block correspond to spatial positions of the neighboring video blocks of the current video block relative to a spatial position of the current video block

48. The method of clause 45, further comprising:

-   -   in response to determining that reconstructions of the         neighboring video blocks is based, at least in part, on a coding         mode of the current video block, disabling derivation of the         second set of color component values of the current video block.

49. The method of clause 45, further comprising:

-   -   in response to determining that the current video block is an         inter-coded block and not a combined inter and intra         prediction-coded block, and a block neighboring a corresponding         luma block of the current video block is an intra-coded block, a         combined inter and intra prediction (CIIP) block, or an intra         block copy (IBC) coded block, disabling derivation of the second         set of color component values of the current video block.

50. The method of clause 45, further comprising:

-   -   in response to determining that the current video block is an         inter-coded block and not a combined inter and intra         prediction-coded block, and a block neighboring a corresponding         luma block of the current video block is an intra-coded block, a         combined inter and intra prediction (CIIP) block, or an intra         block copy (IBC) coded block, enabling derivation of the second         set of color component values of the current video block,         wherein the first set of color component values are fixed         values.

51. The method of clause 45, wherein the first set of color component values are fixed values.

52. The method of clause 51, wherein the fixed values correspond to a piecewise index of the linear model of the video coding step.

53. The method of any one or more of clauses 1-52, wherein the neighboring samples may be adjacent or non-adjacent to the current video block.

54. The method of any one or more of clauses 1-52, wherein the neighboring samples may be associated with chroma blocks neighboring the current video block or chroma blocks neighboring the current video block.

55. The method of any one or more of clauses 1-54, wherein the current video block corresponds to a chroma block, the collocated luma block, or the corresponding luma block covering the top-left luma sample of the collocated chroma block.

56. The method of any one or more of clauses 1-54, wherein the second set of color component values of the current video block are stored for use in connection with one or more other video blocks.

57. The method of any one or more of clauses 1-56, wherein the linear model corresponds to a cross-component linear model (CCLM) and the video coding step corresponds to a luma mapping with chroma scaling (LMCS) mode.

58. The method of any one or more of clauses 1-57, wherein the current video block is an inter-coded block, a bi-prediction coded block, a combined inter and intra prediction (CIIP) block, or an intra block copy (IBC) coded block.

59. The method of any one or more of clauses 1-58, wherein the second set of color component values of the current video block are stored for use associated with other video blocks in the bitstream representation.

60. The method of clause 59, wherein the second set of color component values of the current video block are stored in a line buffer for selective availability or unavailability by a neighboring video block included in the other video blocks, wherein the current video block and the neighboring video block are associated with different slices, tiles, tile groups, coding tree units, or rows of coding tree units.

61. The method of clause 60, wherein the second set of color component values of the current video block are fixed values.

62. The method of clause 60, wherein the second set of color component values of the current video block are prevented from derivation.

63. The method of clause 59, wherein the derivation of the second set of color component values of the current video block is prevented.

64. The method of clause 59, wherein the second set of color component values of the current video block are stored in a table for selective availability or unavailability by a neighboring video block included in the other video blocks.

65. The method of clause 64, wherein the second set of color component values of the current video block are dynamically updated.

66. The method of clause 65, wherein the second set of color component values of the current video block are same as a previous video block in the other video blocks.

67. The method of clause 64, wherein the second set of color component values of the current video block are dynamically updated in a first in first out (FIFO) manner.

68. The method of any one or more of clauses 1-67, wherein the first set of color component values correspond to luma sample values and the second set of color component values correspond to chroma scaling factors.

69. A method for video processing, comprising: determining, based on a rule, an applicability of a cross-component codec tool for a conversion between a video block comprising at least two component blocks and a bitstream representation of the video block; and performing the conversion by selectively using the cross-component codec tool.

70. The method of clause 69, wherein the rule is based on a dimension of the current video block.

71. The method of any of clauses 69-70, wherein the rule is based on a partition information of the current video block.

72. The method of any of clauses 69-71, wherein the rule is based on a partition tree structure of the current video block.

73. The method of clause 72, wherein the partition tree structure is one of a dual tree or a single tree.

74. The method of clause 72, wherein the rule specifies to disable the cross-component codec tool due to the current video block being a chroma block with the dual tree partition tree structure.

75. The method of any of clauses 69-74, wherein the rule specifies omitting signaling of syntax elements of the cross-component codec tool in the bitstream representation.

Additional examples and aspects of clauses 69-75 are described in item 15 of section 4.

76. An apparatus in a video system comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to implement the method in any one of clauses 1 to 75.

77. A computer program product stored on a non-transitory computer readable media, the computer program product including program code for carrying out the method in any one of clauses 1 to 75.

FIG. 12 is a block diagram showing an example video processing system 1200 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of the system 1200. The system 1200 may include input 1202 for receiving video content. The video content may be received in a raw or uncompressed format, e.g., 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format. The input 1202 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interface include wired interfaces such as Ethernet, passive optical network (PON), etc. and wireless interfaces such as Wi-Fi or cellular interfaces.

The system 1200 may include a coding component 1204 that may implement the various coding or encoding methods described in the present document. The coding component 1204 may reduce the average bitrate of video from the input 1202 to the output of the coding component 1204 to produce a coded representation of the video. The coding techniques are therefore sometimes called video compression or video transcoding techniques. The output of the coding component 1204 may be either stored, or transmitted via a communication connected, as represented by the component 1206. The stored or communicated bitstream (or coded) representation of the video received at the input 1202 may be used by the component 1208 for generating pixel values or displayable video that is sent to a display interface 1210. The process of generating user-viewable video from the bitstream representation is sometimes called video decompression. Furthermore, while certain video processing operations are referred to as “coding” operations or tools, it will be appreciated that the coding tools or operations are used at an encoder and corresponding decoding tools or operations that reverse the results of the coding will be performed by a decoder.

Examples of a peripheral bus interface or a display interface may include universal serial bus (USB) or high definition multimedia interface (HDMI) or Displayport, and so on. Examples of storage interfaces include serial advanced technology attachment (SATA), peripheral component interconnect (PCI), integrated drive electronics (IDE) interface, and the like. The techniques described in the present document may be embodied in various electronic devices such as mobile phones, laptops, smartphones or other devices that are capable of performing digital data processing and/or video display.

FIG. 13 is a flowchart for an example for a visual media processing method. Steps of this flowchart are described in connection with example 12g(ii) in Section 4 of this document. At step 1302, the process, in response to determining availability of neighboring luma blocks of a corresponding luma block of visual media data, derives a first parameter for a current video chroma block of the visual media data such that the first parameter is derived as a fixed value or a rounding-based average value of the neighboring luma samples of the current video chroma block, wherein the corresponding luma block covers a location associated with the current video chroma block. At step 1304, the process determines a chroma scaling factor based on the first parameter. At step 1306, the process performs a chroma residual scaling of the current video chroma block based on the chroma scaling factor. At step 1308, the process uses the chroma scaling factor for a conversion between the current video chroma block and a bitstream.

FIG. 14 is a flowchart for an example for a visual media processing method. Steps of this flowchart are described in connection with example 12d in Section 4 of this document. At step 1402, the process uses a rule that specifies representing, based on a fixed value, a chroma residual scaling factor for at least one chroma block associated with a current video unit of visual media data, wherein the fixed value is based at least on one or more conditions. At step 1404, the process performs a conversion between the current video unit and a bitstream representation of the current video unit.

FIG. 15 is a flowchart for an example for a visual media processing method. Steps of this flowchart are described in connection with example 15 in Section 4 of this document. At step 1502, during a conversion between a current video block of visual media data and a bitstream representation of the current video block, the process makes a determination of selectively enabling or disabling application of a cross-component linear model (CCLM) on color components of the current video block, based at least in part on dimensions and/or a partition information and/or a coding structure type associated with the current video block, wherein, in the CCLM, a first set of color components are predicted based on a second set of color components.

Some embodiments of the present document are now presented in clause-based format.

X1. A method for visual media processing:

in response to determining availability of neighboring luma blocks of a corresponding luma block of visual media data, deriving a first parameter for a current video chroma block of the visual media data such that the first parameter is derived as a fixed value or a rounding-based average value of the neighboring luma samples of the current video chroma block, wherein the corresponding luma block covers a location associated with the current video chroma block;

determining a chroma scaling factor based on the first parameter:

performing a chroma residual scaling of the current video chroma block based on the chroma scaling factor; and

using the chroma scaling factor for a conversion between the current video chroma block and a bitstream representation of the current video chroma block.

X2. The method of clause X1, wherein determining the chroma scaling factor based on the first parameter comprises:

using the first parameter to identify an index of a piecewise linear function.

X3. The method of clause X2, further comprising:

upon identifying the index of the piecewise linear function, determining the chroma residual factor used in the chroma residual scaling of the current video chroma block.

X4. The method of clause X1, wherein the chroma residual scaling factor is included as a field in the bitstream representation.

X5. The method of any one or more of clauses X1-X4, wherein, if at least one of the neighboring luma blocks is available, the first parameter is derived as the rounding-based average value of the neighboring luma samples of the current video chroma block.

X6. The method of any one or more of clauses X1-X4, wherein, if neighboring luma blocks are unavailable, the first parameter is derived as the fixed value.

X7. The method of clause X1 or X6, wherein the fixed value is zero.

X8. The method of clause X1 or X6, wherein the fixed value is based on a bit depth of a luma color component and/or chroma color components.

X9. The method of clause X8, wherein the fixed value is (1<<BitDepth)−1, or 1<<(BitDepth−1), where BitDepth denotes the bit depth of the luma color components and/or chroma color components.

X10. The method of clause X6, wherein the fixed value is a maximum bin index in the piecewise linear function.

X11. The method of clause X6, wherein the fixed value is a minimum bin index in the piecewise linear function.

X12. The method of clause X6, wherein the fixed value is an average of a minimum bin index and a maximum bin index in the adaptive piecewise linear function.

A1. A method for visual media processing:

using a rule that specifies representing, based on a fixed value, a chroma residual scaling factor for at least one chroma block associated with a current video unit of visual media data, wherein the fixed value is based at least on one or more conditions; and

performing a conversion between the current video unit and a bitstream representation of the current video unit.

A2. The method of clause A1, wherein the one or more conditions include:

(1) a type of picture/slice/tile group/tile/subpicture/coding tree unit (CTU) row/CTU/virtual pipeline data unit (VPDU),

(2) a width and/or a height of picture/slice/tile group/tile/subpicture/CTU row/CTU/VPDU,

(3) a coding mode of the current video unit, or

(4) neighboring samples of the current video unit.

A3. The method of clause A2, wherein a first fixed value is computed for a picture/slice/tile group/tile/subpicture/CTU row/CTU/VPDU of a first type and a second fixed value is computed for a picture/slice/tile group/tile/subpicture/CTU row/CTU/VPDU of a second type.

B1. A method for visual media processing, comprising:

during a conversion between a current video block of visual media data and a bitstream representation of the current video block, making a determination of selectively enabling or disabling application of a cross-component linear model (CCLM) on color components of the current video block, based at least in part on dimensions and/or a partition information and/or a coding structure type associated with the current video block, wherein, in the CCLM, a first set of color components are predicted based on a linear combination of a second set of color components.

B2. The method of clause B1, wherein the coding structure type is a single tree or a dual tree.

B3. The method of clause B1, wherein the CCLM is disabled based on the current video block being a chroma block and the partition information specifying a dual-tree partition.

B4. The method of clause B1, wherein the CCLM is disabled based on the dimensions of the current video block failing to meet at least one threshold condition.

B5. The method of clause B1, wherein the at least one threshold condition is based on a mathematical relationship between (i) a width of the current video block and/or a height of the current video block and (ii) one or more threshold parameters.

B6. The method of clause B1, wherein the one or more threshold parameters have values of 8, 16, 64, or 4096.

B7. The method of clause B1, wherein the CCLM is disabled when the partition information specifies that the current video block is at a leaf node with horizontal split, and a collocated luma block associated with the current video block is at a leaf node with vertical split.

B8. The method of clause B7, wherein the CCLM is disabled when the current video block is a chroma block with a horizontal ternary split, and the collocated luma block is with a vertical ternary split.

B9. The method of clause B7, wherein the CCLM is disabled when the current video block is a chroma block split with a horizontal binary split, and the collocated luma block is with a vertical ternary split.

B10. The method of clause B7, wherein the CCLM is disabled when the current video block is a chroma block split with a horizontal binary split, and the collocated luma block is with a vertical ternary split.

B11. The method of clause B7, wherein the CCLM is disabled when the current video block is a chroma block split with a horizontal binary split, and the collocated luma block is with a vertical binary split.

B12. The method of clause B1, wherein the CCLM is disabled when the partition information specifies that the current video block is at a leaf node with vertical split, and a collocated luma block associated with the current video block is at a leaf node with horizontal split.

B13. The method of clause B12, wherein the CCLM is disabled when the current video block is a chroma block with a vertical ternary split, and the collocated luma block is with a horizontal ternary split.

B14. The method of clause B12, wherein the CCLM is disabled when the current video block is a chroma block with a vertical ternary split, and the collocated luma block is with a horizontal binary split.

B15. The method of clause B12, wherein the CCLM is disabled when the current video block is a chroma block with a binary vertical split, and the collocated luma block is a horizontal ternary split.

B16. The method of clause B1, wherein the CCLM is disabled when the current video block is a chroma block with a vertical binary split, and the collocated luma block is with a horizontal binary split.

B17. The method of any one or more of clauses B1-B16, further comprising:

upon determining that the CCLM is disabled for the current video block, excluding, from the bitstream representation, a syntax element indicative of application of the CCLM.

B18. The method of any one or more of clauses B1-B16, further comprising:

upon determining that the CCLM is disabled for the current video block, including, in the bitstream representation, a syntax element indicative of application of the CCLM, wherein usage of one or more prediction modes associated with the CCLM is skipped.

B19. The method of any one or more of clauses B1-B16, further comprising:

upon determining that the CCLM is disabled for the current video block, including, in the bitstream representation, a syntax element indicative of application of the CCLM, a default prediction mode is applied.

B20. The method of clause B19, wherein the default prediction mode is planar mode or DC mode.

B21. The method of any one or more of claims X1-B20, wherein the conversion includes generating the bitstream representation from a video block.

B22. The method of any one or more of claims X1-B20, wherein the conversion includes generating pixel values of a video block from the bitstream representation.

B23. A video encoder apparatus comprising a processor configured to implement a method recited in any one or more of clauses X1-B20.

B24. A video decoder apparatus comprising a processor configured to implement a method recited in any one or more of clauses X1-B20.

B25. A computer readable medium having code stored thereon, the code embodying processor-executable instructions for implementing a method recited in any one or more of clauses X1-B20.

In the present document, the term “video processing” or “visual media processing” may refer to video encoding, video decoding, video compression or video decompression. For example, video compression algorithms may be applied during conversion from pixel representation of a video to a corresponding bitstream representation or vice versa. The bitstream representation of a current video block may, for example, correspond to bits that are either co-located or spread in different places within the bitstream, as is defined by the syntax. For example, a macroblock may be encoded in terms of transformed and coded error residual values and also using bits in headers and other fields in the bitstream. Furthermore, during conversion, a decoder may parse a bitstream with the knowledge that some fields may be present, or absent, based on the determination, as is described in the above solutions. Similarly, an encoder may determine that certain syntax fields are or are not to be included and generate the coded representation accordingly by including or excluding the syntax fields from the coded representation.

From the foregoing, it will be appreciated that specific embodiments of the presently disclosed embodiments have been described herein for purposes of illustration, but that various modifications may be made without deviating from the scope of the disclosure. Accordingly, the presently disclosed embodiments are not limited except as by the appended claims.

Implementations of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term “data processing unit” or “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an field programmable gate array (FPGA) or an application specific integrated circuit (ASIC).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., erasable programmable read-only memory (EPROM), electrically EPROM (EEPROM), and flash memory devices. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

It is intended that the specification, together with the drawings, be considered exemplary only, where exemplary means an example. As used herein, the use of “or” is intended to include “and/or”, unless the context clearly indicates otherwise.

While this patent document contains many specifics, these should not be construed as limitations on the scope of any disclosure or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of the present disclosure. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document. 

What is claimed is:
 1. A method of processing video data, comprising: determining, for a conversion between a chroma block of a video and a bitstream of the video, whether to enable or disable a cross-component linear model intra prediction on the chroma block, based on partition information and a block dimension of a luma video unit and a chroma video unit, wherein the luma video unit covers at least one sample of a corresponding luma region of the chroma block, wherein the chroma video unit covers at least one sample of a corresponding chroma region, and wherein the chroma block is in a dual tree structure in which a luma component and at least one chroma component have separate partition tree structures; determining, in response to the cross-component linear model intra prediction being enabled, parameters of the cross-component linear model intra prediction based at least on neighboring chroma samples of the chroma block; applying the cross-component linear model intra prediction to derive prediction values of the chroma block based on the parameters; and performing the conversion based on the prediction values.
 2. The method of claim 1, wherein the cross-component linear model intra prediction is disabled when the block dimension of the chroma video unit is 32×32 and the block dimension of the luma video unit is 64×64, and wherein the partition information specifies that the chroma video unit is split with a horizontal split operation that is not a quadtree split operation and the luma video unit is split with a vertical split operation that is not a quadtree split operation.
 3. The method of claim 2, wherein the cross-component linear model intra prediction is disabled when the chroma video unit is split with a horizontal ternary split operation and the luma video unit is split with a vertical ternary split operation.
 4. The method of claim 2, wherein the cross-component linear model intra prediction is disabled when the chroma video unit is split with a horizontal ternary split operation and the luma video unit is split with a vertical binary split operation.
 5. The method of claim 2, wherein the cross-component linear model intra prediction is disabled when the chroma video unit is split with a horizontal binary split operation and the luma video unit is split with a vertical ternary split operation.
 6. The method of claim 2, wherein the cross-component linear model intra prediction is disabled when the chroma video unit is split with a horizontal binary split operation and the luma video unit is split with a vertical binary split operation.
 7. The method of claim 1, wherein the cross-component linear model intra prediction is disabled when the block dimension of the luma video unit is 64×64, and wherein the partition information specifies that the chroma video unit is split with a vertical split operation that is not a quadtree split operation and the luma video unit is split with a horizontal split operation that is not a quadtree split operation.
 8. The method of claim 7, wherein the cross-component linear model intra prediction is disabled when the chroma video unit is split with a vertical ternary split operation and the luma video unit is split with a horizontal ternary split operation.
 9. The method of claim 7, wherein the cross-component linear model intra prediction is disabled when the chroma video unit is split with a vertical ternary split operation and the luma video unit is split with a horizontal binary split operation.
 10. The method of claim 7, wherein the cross-component linear model intra prediction is disabled when the chroma video unit is split with a binary vertical split operation and the luma video unit is split with a horizontal ternary split operation.
 11. The method of claim 7, wherein the cross-component linear model intra prediction is disabled when the chroma video unit is split with a vertical binary split operation and the luma video unit is split with a horizontal binary split operation.
 12. The method of claim 1, further comprising, upon determining that the cross-component linear model intra prediction is disabled, excluding, from the bitstream, a syntax element related to the cross-component linear model intra prediction.
 13. The method of claim 1, wherein the conversion includes encoding the chroma block into the bitstream.
 14. The method of claim 1, wherein the conversion includes decoding the chroma block from the bitstream.
 15. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to: determine, for a conversion between a chroma block of a video and a bitstream of the video, whether to enable or disable a cross-component linear model intra prediction on the chroma block, based on partition information and a block dimension of a luma video unit and a chroma video unit, wherein the luma video unit covers at least one sample of a corresponding luma region of the chroma block, wherein the chroma video unit covers at least one sample of a corresponding chroma region, and wherein the chroma block is in a dual tree structure in which a luma component and at least one chroma component have separate partition tree structures; determine, in response to the cross-component linear model intra prediction being enabled, parameters of the cross-component linear model intra prediction based at least on neighboring chroma samples of the chroma block; apply the cross-component linear model intra prediction to derive prediction values of the chroma block based on the parameters; and perform the conversion based on the prediction values.
 16. The apparatus of claim 15, wherein the cross-component linear model intra prediction is disabled when the block dimension of the chroma video unit is 32×32 and the block dimension of the luma video unit is 64×64, and wherein the partition information specifies that the chroma video unit is split with a horizontal split operation that is not a quadtree split operation and the luma video unit is split with a vertical split operation that is not a quadtree split operation, wherein the cross-component linear model intra prediction is disabled when the block dimension of the luma video unit is 64×64, and wherein the partition information specifies that the chroma video unit is split with a vertical split operation that is not a quadtree split operation and the luma video unit is split with a horizontal split operation that is not a quadtree split operation, and wherein the processor is further caused to, upon determining that the cross-component linear model intra prediction is disabled, excluding, from the bitstream, a syntax element related to the cross-component linear model intra prediction.
 17. A non-transitory computer-readable storage medium storing instructions that cause a processor to: determine, for a conversion between a chroma block of a video and a bitstream of the video, whether to enable or disable a cross-component linear model intra prediction on the chroma block, based on partition information and a block dimension of a luma video unit and a chroma video unit, wherein the luma video unit covers at least one sample of a corresponding luma region of the chroma block, wherein the chroma video unit covers at least one sample of a corresponding chroma region, and wherein the chroma block is in a dual tree structure in which a luma component and at least one chroma component have separate partition tree structures; determine, in response to the cross-component linear model intra prediction being enabled, parameters of the cross-component linear model intra prediction based at least on neighboring chroma samples of the chroma block; apply the cross-component linear model intra prediction to derive prediction values of the chroma block based on the parameters; and perform the conversion based on the prediction values.
 18. The non-transitory computer-readable storage medium of claim 17, wherein the cross-component linear model intra prediction is disabled when the block dimension of the chroma video unit is 32×32 and the block dimension of the luma video unit is 64-64, and wherein the partition information specifies that the chroma video unit is split with a horizontal split operation that is not a quadtree split operation and the luma video unit is split with a vertical split operation that is not a quadtree split operation, wherein the cross-component linear model intra prediction is disabled when the block dimension of the luma video unit is 64×64, and wherein the partition information specifies that the chroma video unit is split with a vertical split operation that is not a quadtree split operation and the luma video unit is split with a horizontal split operation that is not a quadtree split operation, and wherein the processor is further caused to, upon determining that the cross-component linear model intra prediction is disabled, excluding, from the bitstream, a syntax element related to the cross-component linear model intra prediction.
 19. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises: determining, for generating the bitstream, whether to enable or disable a cross-component linear model intra prediction on the chroma block, based on partition information and a block dimension of a luma video unit and a chroma video unit, wherein the luma video unit covers at least one sample of a corresponding luma region of the chroma block, wherein the chroma video unit covers at least one sample of a corresponding chroma region, and wherein the chroma block is in a dual tree structure in which a luma component and at least one chroma component have separate partition tree structures; determining, in response to the cross-component linear model intra prediction being enabled, parameters of the cross-component linear model intra prediction based at least on neighboring chroma samples of the chroma block; applying the cross-component linear model intra prediction to derive prediction values of the chroma block based on the parameters; and generating the bitstream based on the prediction values.
 20. The non-transitory computer-readable recording medium of claim 19, wherein the cross-component linear model intra prediction is disabled when the block dimension of the chroma video unit is 32-32 and the block dimension of the luma video unit is 64×64, and wherein the partition information specifies that the chroma video unit is split with a horizontal split operation that is not a quadtree split operation and the luma video unit is split with a vertical split operation that is not a quadtree split operation, wherein the cross-component linear model intra prediction is disabled when the block dimension of the luma video unit is 64×64, and wherein the partition information specifies that the chroma video unit is split with a vertical split operation that is not a quadtree split operation and the luma video unit is split with a horizontal split operation that is not a quadtree split operation, and wherein the method further comprises, upon determining that the cross-component linear model intra prediction is disabled, excluding, from the bitstream, a syntax element related to the cross-component linear model intra prediction. 